Surgical systems and methods for facilitating ad-hoc intraoperative planning of surgical procedures

ABSTRACT

A surgical system and method for facilitating ad-hoc intraoperative planning of a surgical step to be performed at a target site, the system comprising a digitization device, a navigation system, and a computing device. The digitization device intraoperatively facilitates establishment of one or more local virtual references relative to the target site. The navigation system tracks states of the digitization device. The computing device is coupled to the navigation system and comprises one or more processors and a non-transitory storage medium having stored thereon a computer-aided design (CAD) program. When executed by the one or more processors, the CAD program is configured to generate a virtual reference frame, register the one or more local virtual references within the virtual reference frame, and enable arrangement of different geometrical design objects within the virtual reference frame relative to one or more registered local virtual references to intraoperatively plan the surgical step.

CROSS-REFERENCE TO RELATED APPLICATIONS

The subject patent application claims priority to and all the benefitsof U.S. Provisional Patent Application No. 62/485,779 filed on Apr. 14,2017, and U.S. Provisional Patent Application No. 62/502,414 filed onMay 5, 2017, the disclosures of which are hereby incorporated byreference in their entirety.

TECHNICAL FIELD

The present disclosure relates, generally, to surgical systems and, morespecifically, to surgical systems and methods for facilitating ad-hocintraoperative planning of surgical procedures.

BACKGROUND

The workflow associated with many conventional surgical proceduresgenerally comprises three phases: diagnosis, planning, and execution.With the exception of time-sensitive emergency surgical interventions(e.g., treating traumatic injuries), the diagnosis and planning phasestend to be performed preoperatively and are followed by intraoperativeexecution. During the planning stage, a surgeon generally identifies orotherwise formulates discrete surgical steps to be performed duringexecution (often in a predetermined, ideal sequence). It will beappreciated that the planning phase can be highly subjective, andtypically involves the surgeon taking a number of different factors intoconsideration, including without limitation: the type of surgicalprocedure to be executed; the types of tools, instrumentation, andprostheses which are available; the surgeon's preferences, training, andexperience: the patient's condition, anatomy, and medical history; andthe amount of time available before execution is to occur.

In many surgical procedures, the diagnosis and planning phases bothfrequently involve the use of preoperatively-acquired imaging (e.g., viaX-ray, CT-scan, MRI, and the like). By way of illustrative example, aphysician could diagnose a patient's condition (e.g., osteoarthritis)with an X-ray image acquired at a target site (e.g., the patient's hipjoint), and an appropriate surgical procedure to be executed (e.g.,total hip arthroplasty) could then be planned by a surgeon based on MRIscans of the patient's anatomy at the target site. Conventionally, thesurgeon will formulate certain steps of the surgical plan based onmeasurements taken from one or more 2D images preoperatively-acquired atthe target site.

By way of illustrative example, preoperative planning one or more typesof orthopedic surgical interventions may involve surgical steps definedby the surgeon “tracing” or “marking up” an X-ray image of the patient'sbone. Here, planning surgical steps may include drawing points atspecific anatomical landmarks, drawing lines between or with respect toanatomical landmarks, taking measurements (e.g., distances and angles)between various lines and points that have been drawn, creatingadditional points and lines based on those measurements, and the like.Formulating these types of preoperative planning steps allows thesurgeon to subsequently execute the surgical procedure in apatient-specific manner, whereby lines “traced” on the X-ray mayrepresent or otherwise correspond to surgical steps to be made duringexecution (e.g., cutting bone in a way that corresponds to a line tracedover the X-ray).

Advantageously, the surgical steps formulated with preoperative planningare executed as intended during the surgical procedure. However, it willbe appreciated that postoperative results may not match a preoperativesurgical plan for various reasons (e.g., human error during planning,unexpected complications during execution, variations in the patient'sanatomy, and the like).

Execution of the surgical plan generally involves the surgeon utilizingone or more different types of surgical tools (e.g., saws, drills,milling devices, and the like) to facilitate approaching, manipulating,or otherwise effecting treatment of the target site. By way ofillustration, surgical tools are commonly used in orthopedic surgicalprocedures which involve the correction, stabilization, resection, orreplacement of one or more parts of the patient's anatomy, such as tohelp improve patient mobility, reduce pain, mitigate the risk ofsubsequent injury or damage, and the like.

In certain surgical procedures, a navigation system (or, “trackingsystem”) may also be utilized during execution of the surgical plan inorder to assist surgeons in, guiding, positioning, and/or movingsurgical tools, instrumentation, prostheses, hardware, and the likerelative to the target site with enhanced accuracy and precision. Tothis end, the navigation system generally tracks states of the surgicaltool and also tracks states of one or more patient trackers attached tothe patient's anatomy relative to the target site, both of which maymove during the surgical procedure. Navigation systems are used inconnection with both hand-held surgical tools and surgical tools whichare coupled to robotic manipulators. In some applications, thenavigation system may also be configured to relate portions of thepatient's anatomy to 3D renderings preoperatively-acquired at the targetsite in order to define virtual boundaries used to constrain thesurgical tool to desired areas.

While the use of navigation systems, navigated surgical tools, and/orcomputer-aided preoperative planning affords opportunities for executingsurgical procedures with enhanced accuracy and precision, it will beappreciated that the success of a surgical procedure depends on bothprecise preoperative planning and accurate reproduction of the planduring execution. In order for navigation systems and/or navigatedsurgical tools to work as intended, significant reliance is generallyplaced on preoperative planning and/or predefined workflows forexecuting certain types of surgical procedures, which may make itdifficult for the surgeon to deviate from the preoperative plan duringexecution. Furthermore, it will be appreciated that certain surgicalprocedures are less suitable for the use of computer-aided preoperativeplanning, navigation systems, and/or navigated surgical tools, such aswhere the target site is difficult to approach, involves complex tissuegeometry, involves a revision of a previous procedure, or involves anemergency surgical intervention. As such, many types of surgicalprocedures are routinely carried “manually” without the benefitsafforded by computer-aided planning, navigation systems, and/ornavigated surgical tools.

Additionally, conventional techniques do not afford the surgeon theability to intraoperatively plan surgical steps “on the fly” usingcomputer-aided technology. The resources (e.g., software) used forpreoperative planning are no longer available, cannot be practicallyincorporated, or otherwise are incapable of providing any meaningfuloutput during surgery. This limits the surgeon's ability to make ad-hocadjustments or customized plans in the operating room.

Computer-aided preoperative planning also restricts the workflow bywhich the surgeon is able to plan a surgical step. For example, planningprograms may limit how an implant is shaped, how/where the implant canbe defined relative to the anatomical model, and so on. Thus,conventional planning approaches fail to provide the untethered creativefreedom to enable surgeons to plan the surgery as they see fit.

Moreover, computer-aided preoperative planning requires a significantamount of resources and time from the surgeon, and others. For thesereasons, many surgeons maintain a manual approach to surgery that doesnot involve the complexity of computer-aided preoperative planning. Thestate of the art provides no “middle ground” alternative for suchsurgeons.

Accordingly, there remains a need in the art to address at least theaforementioned issues.

SUMMARY

The present disclosure provides a surgical system for facilitatingad-hoc intraoperative planning of a surgical step to be performed at atarget site. The surgical system comprises a digitization device, anavigation system, and a computing device. The digitization device isconfigured to intraoperatively facilitate establishment of one or morelocal virtual references relative to the target site. The navigationsystem is configured to track states of the digitization device. Thecomputing device is coupled to the navigation system and comprises oneor more processors and a non-transitory storage medium having storedthereon a computer-aided design (CAD) program. When executed by the oneor more processors, the CAD program is configured to generate a virtualreference frame, register the one or more local virtual referenceswithin the virtual reference frame, and enable arrangement of differentgeometrical design objects within the virtual reference frame relativeto one or more registered local virtual references to intraoperativelyplan the surgical step.

The present disclosure also provides a computer-assisted method forfacilitating ad-hoc intraoperative planning of a surgical step to beperformed at a target site using a surgical system comprising adigitization device, a navigation system, and a computing device coupledto the navigation system. The computing device comprises one or moreprocessors and a non-transitory storage medium having stored thereon acomputer-aided design (CAD) program being executable by the one or moreprocessors. The method comprises: tracking states of the digitizationdevice with the navigation system and intraoperatively establishing oneor more local virtual references relative to the target site with thedigitization device. The method also comprise generating a virtualreference frame with the CAD program, registering the one or more localvirtual references within the virtual reference frame with the CADprogram, and arranging different geometrical design objects within thevirtual reference frame relative to one or more registered local virtualreferences with the CAD program to intraoperatively plan the surgicalstep.

The present disclosure also provides a computer-aided design (CAD)program being stored on a non-transitory storage medium and configuredto facilitate ad-hoc intraoperative planning of a surgical step to beperformed at a target site. When executed by one or more processors, theCAD program is configured to: generate a virtual reference frame;register one or more local virtual references established relative tothe target site within the virtual reference frame; and enablearrangement of different geometrical design objects within the virtualreference frame relative to one or more registered local virtualreferences to intraoperatively plan the surgical step.

Other features and advantages of the present disclosure will be readilyappreciated, as the same becomes better understood, after reading thesubsequent description taken in conjunction with the accompanyingdrawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view of a surgical system for facilitatingad-hoc intraoperative planning of a surgical step to be performed at atarget site, the surgical system shown comprising a navigation system, adigitization device, a surgical tool, a head-mountable display (HMD)unit, a patient tracker attached to a patient's anatomy, a controlinput, and a computing device according to one embodiment of the presentdisclosure.

FIG. 2 is a block diagram illustrating general communication between thecomputing device and other components of the surgical system of FIG. 1,the computing device shown having a memory device with a computer-aideddesign (CAD) program stored thereon according to one embodiment of thepresent disclosure.

FIG. 3 is a block diagram illustrating an example software architectureof the CAD program of FIG. 2 according to one embodiment of the presentdisclosure.

FIG. 4 is a schematic diagram illustrating example interactions betweenthe navigation system, the digitization device, the surgical tool, theHMD unit, the patient tracker, the control input, and the CAD program ofthe computing device of FIG. 1.

FIG. 5 is a block diagram illustrating aspects of a graphical userinterface (GUI) for use in arranging different geometrical designobjects within a virtual reference frame with the CAD program of FIG. 2to plan a surgical step according to one embodiment of the presentdisclosure.

FIG. 6 depicts different types of geometrical design objects which canbe arranged within a virtual reference frame of the CAD program of FIG.2.

FIG. 7 depicts different aspects of the CAD program of FIG. 2 which canbe used to facilitate arranging geometrical design objects within avirtual reference frame.

FIG. 8 depicts different algorithms which can be used by the CAD programof FIG. 2 to facilitate arranging geometrical design objects within avirtual reference frame.

FIG. 9A is a visualization of a virtual reference frame rendered withthe CAD program of FIG. 2, shown depicting an example geometrical designobject having an object index illustrating a pose within the virtualreference frame, with the geometrical design object realized as arectangular plane having a span.

FIG. 9B is another visualization of the virtual reference frame of FIG.9A, shown depicting the geometrical design object as having beenadjusted to an octagonal plane while maintaining the same span and thesame pose within the virtual reference frame.

FIG. 10A is another visualization of the virtual reference frame and thegeometrical design object of FIG. 9B, shown depicting options foradjusting the pose of the geometrical design object in six degrees offreedom relative to its object index within the virtual reference frame.

FIG. 10B is another visualization of the virtual reference frame and thegeometrical design object of FIG. 10A, shown depicting the geometricaldesign object in a different pose within the virtual reference frameafter having been rotated about its object index.

FIG. 11A is another visualization of the virtual reference frame and thegeometrical design object of FIG. 10B, shown depicting the geometricaldesign object with the span shown in FIGS. 9A-10A.

FIG. 11B is another visualization of the virtual reference frame and thegeometrical design object of FIG. 11A, shown depicting the geometricaldesign object in the same pose as illustrated in FIG. 11A but havingbeen adjusted to a larger span.

FIG. 12 is a visualization of a virtual reference frame rendered withthe CAD program of FIG. 2, shown depicting an example geometrical designobject realized as a line segment defined by two registered localvirtual references established within the virtual reference frame, andanother example geometrical design object realized as an octagonal planehaving an object index arranged at one of the registered local virtualreferences of the line segment.

FIG. 13 is another visualization of the virtual reference frame and thegeometrical design objects illustrated in FIG. 12, shown depicting theoctagonal plane arranged perpendicular to the line segment.

FIG. 14 is a visualization of a virtual reference frame rendered withthe CAD program of FIG. 2, shown depicting two example geometricaldesign objects realized as octagonal planes arranged parallel to eachother within the virtual reference frame.

FIG. 15 is a visualization of a virtual reference frame rendered withthe CAD program of FIG. 2, shown depicting an example geometrical designobject realized as an octagonal plane arranged within the virtualreference frame, and three other example geometrical design objectsrealized as points arranged as projected onto the octagonal plane.

FIG. 16 is a visualization of a virtual reference frame rendered withthe CAD program of FIG. 2, shown depicting an example geometrical designobject realized as an octagonal plane arranged within the virtualreference frame, and another example geometrical design object realizedas a line arranged as passing through the octagonal plane.

FIG. 17 is another visualization of the virtual reference frame depictedin FIG. 16, depicting a different geometrical design object constructedvia cutting the octagonal plane along the line as depicted in FIG. 16.

FIG. 18 is an enlarged partial perspective view of components of thesurgical system of FIG. 1, illustrating transforms between a localizerof the navigation system and trackers of the digitization device, thesurgical tool, the HMD unit, and the patient tracker shown attached tothe patient's femur.

FIG. 19A is a partial perspective view of the localizer, thedigitization device, the patient tracker, and the femur of FIG. 18 shownadjacent to a visualization of a virtual reference frame rendered withthe CAD program of FIG. 2, the visualization illustrating transformsbetween a CAD coordinate system and virtual representations of thepatient tracker and the digitization device rendered within thevisualization based on tracked states of the digitization device and thepatient tracker, and further shown illustrating transforms between theCAD coordinate system and a registered local virtual referencepreviously-established as a point representing the saddle point landmarkof the femur.

FIG. 19B is another partial perspective view of the components of thesurgical system and the visualization of the virtual reference frame ofFIG. 19A, shown with the digitization device positioned at thetrochanter minor landmark of the femur, and illustrating registration ofanother local virtual reference within the virtual reference frameestablished as a point representing the trochanter minor landmark of thefemur.

FIG. 19C is another partial perspective view of the components of thesurgical system and the visualization of the virtual reference frame ofFIG. 19B, shown illustrating movement of the digitization devicerelative to the femur and also illustrating corresponding movement ofthe virtual representation of the digitization device relative to theregistered local virtual references arranged within the virtualreference frame, the registered local virtual references having beenestablished as points representing the trochanter minor landmark and thesaddle point landmark of the femur.

FIG. 19D is another partial perspective view of the components of thesurgical system and the visualization of the virtual reference frame ofFIG. 19C, shown illustrating transforms of one example geometricaldesign object realized as a line segment defined by two additionalregistered local virtual references established within the virtualreference frame, and shown illustrating transforms of another examplegeometrical design object realized as an octagonal plane arranged withone edge fixed relative to the registered local virtual references ofthe line segment, the octagonal plane further shown having an objectindex fixed at another registered local virtual reference establishedwithin the virtual reference frame.

FIG. 19E is another partial perspective view of the components of thesurgical system and the visualization of the virtual reference frame ofFIG. 19D, shown illustrating transforms of an example compoundgeometrical design object realized as a wedge constructed with anotheroctagonal plane arranged at a common edge fixed relative to theregistered local virtual references of the line segment and having anobject index fixed at a calculated virtual reference established byadjusting an angle measured between the octagonal planes.

FIG. 19F is a partial perspective view of the components of the surgicalsystem and the visualization of the virtual reference frame of FIG. 19E,shown with the femur and the tracker depicted as being viewed throughthe HMD unit of FIG. 18 to illustrate the wedge and the pointsestablished at the saddle point landmark and the trochanter minorlandmark rendered overlaid onto the femur with augmented reality.

FIG. 20A is a representation of one embodiment of the GUI of FIG. 5,shown depicting a visualization of a virtual reference frame adjacent tofunctions of the CAD program including options to create new objects,adjust objects, measure, manage objects, and manage views of thevisualization, shown with an object manager function selected andlisting objects arranged within the virtual reference frame with some ofthe objects hidden in the visualization.

FIG. 20B is another representation of the GUI of FIG. 20A, shown withthe object manager function still selected and listing the objectsarranged within the virtual reference frame with some of the objectsunhidden in the visualization.

FIG. 21A is another representation of the GUI of FIG. 20A, shown withthe view manager function selected and indicating that the visualizationis orientated to a surgeon's view.

FIG. 21B is another representation of the GUI of FIG. 21A, shown withthe view manager function still selected and indicating that thevisualization is orientated to a situs view.

FIG. 22A is another representation of the GUI of FIG. 20A, shown withthe new object function selected to create a point at a pointer tip of avirtual representation of the digitization device rendered within thevisualization.

FIG. 22B is another representation of the GUI of FIG. 22A, shown withthe new object function selected to create the point offset from thepointer tip of the virtual representation of the digitization devicerendered within the visualization.

FIG. 23A is another representation of the GUI of FIG. 20A, shown withthe new object function selected to create a line segment using definedobjects arranged within the virtual reference frame selectable from alist.

FIG. 23B is another representation of the GUI of FIG. 23A, shown withthe new object function still selected to create the line segment usinga point selected from the list, and shown with the line segment fixed tothe point and to a pointer tip of a virtual representation of thedigitization device rendered within the visualization.

FIG. 23C is another representation of the GUI of FIG. 23B, shown withthe new object function still selected to create the line segment usingtwo points selected from the list, and shown with the line segment fixedto both points and unfixed from the pointer tip of the virtualrepresentation of the digitization device rendered within thevisualization.

FIG. 24A is another representation of the GUI of FIG. 20A, shown withthe new object function selected to create an octagonal plane fixed to apointer tip of a virtual representation of the digitization devicerendered within the visualization and arranged in a perpendicularfashion relative to the virtual representation of the digitizationdevice.

FIG. 24B is another representation of the GUI of FIG. 20A, shown withthe new object function selected to create an octagonal plane fixed to apointer tip of a virtual representation of the digitization devicerendered within the visualization and arranged in a parallel fashionrelative to the virtual representation of the digitization device.

FIG. 24C is another representation of the GUI of FIG. 20A, shown withthe new object function selected to create an octagonal plane fixed to apointer tip of a virtual representation of the digitization devicerendered within the visualization and arranged in a parallel and rotatedfashion relative to the virtual representation of the digitizationdevice.

FIG. 25A is another representation of the GUI of FIG. 20A, shown withthe new object function selected to create an octagonal plane shownfixed to a point selected from a list of objects arranged within thevirtual reference frame.

FIG. 25B is another representation of the GUI of FIG. 25A, shown withthe octagonal plane fixed to the point and rotating about the point tofollow movement of a virtual representation of the digitization devicerendered within the visualization, and shown displaying a messageprompting a user to select or digitize additional objects to fullydefine the octagonal plane.

FIG. 25C is another representation of the GUI of FIG. 25B, shown withthe octagonal plane fixed to the point and rotated about the point tofix to another point selected from the list of objects arranged withinthe virtual reference frame, and shown still displaying the messageprompting the user to select or digitize additional objects to fullydefine the octagonal plane.

FIG. 25D is another representation of the GUI of FIG. 25C, shown withthe octagonal plane fixed to a pointer tip of the virtual representationof the digitization device rendered within the visualization, with theoctagonal plane having been fully-defined with a newly-registered localvirtual reference established as another point.

FIG. 26A is another representation of the GUI of FIG. 26A, shown withthe adjust object function selected to construct a compound object fromthe selected octagonal plane arranged within the virtual referenceframe.

FIG. 26B is another representation of the GUI of FIG. 26A, showndepicting the new object function selected to construct the compoundobject from the selected octagonal plane, with the compound objectselected as an osteotomy plane offset from the selected octagonal plane.

FIG. 27 is a partial perspective view of the digitization device, thepatient tracker, and the femur of FIG. 18 shown adjacent to avisualization of a virtual reference frame rendered with the CAD programof FIG. 2, the visualization depicting virtual representations of thepatient tracker and the digitization device rendered within thevisualization based on tracked states of the digitization device and thepatient tracker.

FIG. 28 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 27, shown with a pointer tip of the digitizationdevice positioned along the femoral neck of the femur to digitize andestablish a point cloud about the femoral neck, and shown with thecorresponding point cloud rendered in the visualization.

FIG. 29 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 28, shown with the pointer tip positioned at thesaddle point landmark of the femur to digitize and establish a point atthe saddle point, and shown with the corresponding point rendered in thevisualization.

FIG. 30 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 29, shown with the pointer tip positioned at thetrochanter minor landmark of the femur to digitize and establish a pointat the trochanter minor landmark, and shown with the corresponding pointrendered in the visualization.

FIG. 31 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 30, shown with a line segment rendered withinthe visualization extending between the points established at the saddlepoint landmark and the trochanter minor landmark, shown with the pointertip positioned at the medial epicondyle landmark of the femur todigitize and establish a point at the medial epicondyle landmark, andshown with the corresponding point rendered in the visualization.

FIG. 32 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 31, shown with the pointer tip positioned at thelateral epicondyle landmark of the femur to digitize and establish apoint at the lateral epicondyle landmark, shown with the correspondingpoint rendered in the visualization, and further shown with a linesegment rendered within the visualization extending between the pointsestablished at the medial epicondyle landmark and at the lateralepicondyle landmark.

FIG. 33 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 32, shown with a calculated point renderedwithin the visualization arranged along the line segment equidistantlybetween the points established at the medial epicondyle landmark and atthe lateral epicondyle landmark.

FIG. 34 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 33, shown with another line segment renderedwithin the visualization extending between the calculated point and thepoint established at the saddle point landmark.

FIG. 35 is another partial perspective view of the digitization device,the patient tracker, the femur, and an enlarged view of thevisualization of the virtual reference frame of FIG. 34, shown with anoctagonal plane fixed to the pointer tip of the virtual representationof the digitization device rendered within the visualization andarranged in a parallel fashion relative to the virtual representation ofthe digitization device.

FIG. 36 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 35, shown with digitization device having beenmoved relative to the femur and with the visualization depictingcorresponding movement of the octagonal plane fixed to the pointer tipof the virtual representation of the digitization device.

FIG. 37 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 36, shown with the visualization depicting theoctagonal plane orientated concurrently with the virtual representationof the digitization device and positioned fixed to the point establishedat the saddle point landmark of the femur, the octagonal plane shownorientated at an angle relative to the line segment rendered within thevisualization extending between the calculated point and the pointestablished at the saddle point landmark.

FIG. 38 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 37, shown with the visualization depicting theoctagonal plane having been arranged at a larger angle relative to theline segment while remaining fixed to the point established at thesaddle point landmark of the femur.

FIG. 39 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 38, shown with the visualization depicting theoctagonal plane in the same arrangement at the larger angle, and furthershowing a measured distance between the octagonal plane and the pointestablished at the trochanter minor landmark of the femur.

FIG. 40A is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 39, shown with the visualization depicting theoctagonal plane arranged so at to be at a smaller distance relative tothe point established at the trochanter minor landmark of the femurwhile still at the larger angle relative to the line segment.

FIG. 40B is an enlarged partial perspective view of the patient trackerand the femur of Figure of FIG. 40A shown depicted as being viewedthrough the HMD unit of FIG. 18 to illustrate the octagonal plane andthe point established at the saddle point landmark rendered overlaidonto the femur with augmented reality.

FIG. 41 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 40A, shown with the visualization depicting anosteotomy plane constructed with another, parallel octagonal planespaced at a cut distance.

FIG. 42 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 41, shown with the visualization depicting theosteotomy plane adjusted to a smaller cut distance.

FIG. 43 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 42, shown with the visualization depicting avolume defined by the osteotomy plane.

FIG. 44 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 43, shown with the pointer tip of thedigitization device positioned along the femoral neck of the femur todigitize and establish additional points for in the point cloud aboutthe femoral neck, and shown with the corresponding additional points ofthe point cloud rendered in the visualization.

FIG. 45 is another partial perspective view of the digitization device,the patient tracker, the femur, and the visualization of the virtualreference frame of FIG. 44, shown with the visualization depicting amilling volume calculated by the CAD program from the point cloud andthe constructed osteotomy plane.

FIG. 46A is another partial perspective view of the patient tracker, thefemur, and the visualization of the virtual reference frame of FIG. 45depicted with the surgical tool of FIG. 18 positioned adjacent to thefemoral neck, and shown with a virtual representation of the surgicaltool rendered within the visualization based on tracked states of thesurgical tool, and further shown with the milling volume depicted in thevisualization established as a virtual boundary for the surgical tool.

FIG. 46B is an enlarged partial perspective view of the surgical tool,the patient tracker, and the femur of Figure of FIG. 46A shown depictedas being viewed through the HMD unit of FIG. 18 to illustrate themilling volume and the point established at the saddle point landmarkrendered overlaid onto the femur with augmented reality.

FIG. 47 is another partial perspective view of the surgical tool, thepatient tracker, the femur, and the visualization of the virtualreference frame of FIG. 46A, shown with the surgical tool resecting themilling volume along the femoral neck, and shown with the visualizationdepicting the virtual representation of the surgical tool positionedwithin the virtual boundary.

FIG. 48 is another partial perspective view of the surgical tool, thepatient tracker, the femur, and the visualization of the virtualreference frame of FIG. 47, shown with the surgical tool furtherresecting the milling volume along the femoral neck, and shown with thevisualization depicting the virtual representation of the surgical toolstill position within the virtual boundary.

FIG. 49 is another partial perspective view of the surgical tool, thepatient tracker, the femur, and the visualization of the virtualreference frame of FIG. 48, shown with the surgical tool moved away fromthe femur after resecting the milling volume along the femoral neck toremove the femoral head, and shown with the visualization depicting themilling volume.

DETAILED DESCRIPTION OF THE EMBODIMENTS

With reference now to the drawings, wherein like numerals indicate likeparts throughout the several views, embodiments of a surgical system 100and computer-implemented techniques and methods associated with thesurgical system 100, including a computer-aided design (CAD) program102, are shown throughout (see FIG. 1). As is described in greaterdetail below, the various embodiments of the present disclosure enable auser (e.g., a surgeon) to facilitate ad-hoc intraoperative planning of asurgical step to be performed at a target site TS of a patient's anatomythat is the subject of a surgical procedure.

In FIG. 1, an operating room is shown with a patient undergoing anexample surgical procedure performed using one embodiment of thesurgical system 100 and the CAD program 102 of the present disclosure.In this illustrative example, the target site TS includes portions ofthe patient's femur (F) and tibia (T). However, it will be appreciatedthat the target site TS could comprise any suitable portion of thepatient's anatomy for a given surgical procedure, including other bonesand various other types of tissue. Moreover, it will be appreciated thatthe surgical system 100 and the CAD program 102 described herein may beused in connection with facilitating ad-hoc intraoperative planning ofsurgical steps for a variety of different types of surgical procedures.

As used herein, the term “intraoperative” means occurring, being carriedout, or existing during the execution of a surgical procedure (e.g.,inside an operating room) and, in contrast, the term “preoperative”means occurring, being carried out, or existing before execution of thesurgical procedure (e.g., outside of the operating room). Furthermore,as used herein, the term “ad-hoc” means that one or more steps of thesurgical procedure are planned intraoperatively or “on-the-fly” duringexecution of the surgical procedure (e.g., inside the operating room).

In some embodiments, ad-hoc intraoperative planning may involveformulating improvised surgical steps which deviate from apreoperatively-formulated surgical plan, or may involve formulatingimprovised surgical steps without reliance on apreoperatively-formulated surgical plan. Thus, as is described ingreater detail below, the surgeon can plan one or more surgical stepsintraoperatively using the surgical system 100 and CAD program 102 ofthe present disclosure without necessarily relying onpreoperatively-acquired patient-specific imaging of the target site TS(e.g., a 2D image acquired via X-ray imaging, or a 3D rendering/modelgenerated via MRI imaging segmentation). However, certain aspects andembodiments of the present disclosure could also be used tointraoperatively validate, verify, confirm, or otherwise compliment (oraccommodate deviating from) a preoperatively-formulated surgical planwhich may or may not rely on patient-specific preoperatively-acquiredimaging of the target site TS.

Referring now to FIGS. 1-19F, in order to facilitate ad-hocintraoperative planning of the surgical step, the surgical system 100generally comprises a navigation system 104 (see FIGS. 1-2, 4, and18-19D), a digitization device 106 (see FIGS. 1 and 19A-19D), and acomputing device 108 (see FIGS. 2 and 4). The navigation system 104 isconfigured to, among other things, track states of the digitizationdevice 106 which, in turn, is configured to facilitate intraoperativelyestablishing one or more local virtual references LVR relative to thetarget site TS (see FIGS. 19A-19F). The computing device 108 is coupledto or is otherwise disposed in communication with the navigation system104, and comprises one or more processors 110 and a non-transitorystorage medium, such as a memory device 112, which the CAD program 102is stored on (see FIG. 2).

When executed by the one or more processors 110 of the computing device108, the CAD program 102 is configured to generate a virtual referenceframe VRF (see FIGS. 9A-17 and 19A-19F), to register local virtualreferences LVR within the virtual reference frame VRF, and to enablearrangement of different geometrical design objects GDO (see FIG. 6; seealso FIGS. 5, 7-17, and 19A-19F) within the virtual reference frame VRFrelative to one or more registered local virtual references LVR tointraoperatively plan the surgical step. The configuration of andinteraction between the CAD program 102 and the components of thesurgical system 100 introduced above will be described in greater detailbelow.

Referring now to FIG. 1, the digitization device 106 of the surgicalsystem 100 is movable relative to the target site TS and is configuredfor hand-held operation by the surgeon, or another user of the surgicalsystem 100, to intraoperatively establish one or more local virtualreferences LVR relative to the target site TS which, noted above, can beregistered by the CAD program 102 and used to facilitate planning thesurgical step, as described in greater detail below.

The digitization device 106 illustrated throughout the drawings isrealized as a “straight pointer” and generally comprises a pointer tip114, one or more pointer control inputs 116, a pointer controller 118,and a pointer tracker 120 (depicted schematically in FIG. 1). Thepointer tip 114 is configured to engage against portions of thepatient's anatomy at or about the target site TS (e.g., by touchingagainst bone). The pointer control input 116 is arranged for actuationby the surgeon to, among other things, facilitate establishing the localvirtual references LVR and, in some embodiments, facilitate operatingthe CAD program 102. It will be appreciated that the pointer controlinput 116 of the digitization device 106 could be configured in a numberof different ways sufficient to be actuated by the surgeon (e.g., withbuttons, triggers, switches, knobs, levers, touchscreens, and the like).

In the illustrated embodiment, the pointer control input 116communicates with the pointer controller 118 which, in turn,communicates with the computing device 108, the navigation system 104,and/or other components of the surgical system 100, such as via physicalelectrical connections (e.g., a tethered wire harness) and/or via one ormore types of wireless communication (e.g., with a WiFi™ network,Bluetooth®, a radio network, and the like). It will be appreciated thatthe pointer controller 118 may be configured in a number of differentways and could comprise and/or support various types of electricalcomponents (e.g., sensors, processors, integrated circuits,transceivers, and the like) in some embodiments. In the representativeembodiment illustrated herein, the pointer tracker 120 is firmly affixedto the digitization device 106 and enables the navigation system 104 totrack states of the digitization device 106, as noted above and as isdescribed in greater detail below. In some embodiments, the digitizationdevice 106 may define a predetermined digitizer reference point DRPwhich represents a relative position of the pointer tip 114 within adigitizer coordinate system DCS defined by the position and orientationof the pointer tracker 120 relative to the pointer tip 114 (see FIG.18). Thus, the term “pointer tip 114” and the term “digitizer referencepoint DRP” may be used interchangeably herein.

In some embodiments, the digitization device 106 could be configuredsimilar to as is shown in U.S. Pat. No. 7,725,162, entitled, “SurgerySystem,” the disclosure of which is hereby incorporated by reference.However, it will be appreciated that the digitization device 106 couldbe configured in other ways in some embodiments. Moreover, and as willbe appreciated from the subsequent description below, the surgicalsystem 100 could comprise additional and/or differently-configureddigitization devices 106 in some embodiments, such as to facilitateestablishing local virtual references LVR at the target site TS in otherways (e.g., a hand-held ultrasonic scanner which may not directlycontact portions of the target site TS). Other configurations arecontemplated.

Those having ordinary skill in the art will appreciate that manyconventional surgical procedures involve removing or otherwise treatingtissue of the patient's anatomy (e.g., resecting, cutting, milling,coagulating, lesioning, and the like). In some surgical procedures,portions of the patient's anatomy are removed and replaced by one ormore prosthetic implants (e.g., hip and knee joint implants). Variousprosthetic implants are shown in U.S. Patent Application Publication No.2012/0030429, entitled, “Prosthetic Implant and Method of Implantation,”the disclosure of which is hereby incorporated by reference.

In order to facilitate removing or otherwise treating tissue of thepatient's anatomy, the surgical system 100 may comprises one or moretypes of surgical tools 122 in some embodiments. The surgical tool 122can be used manually by the surgeon during execution of the surgicalprocedure, or, in some embodiments, the surgical system 100 may allowthe surgeon to navigate the surgical tool 122 with respect togeometrical design objects GDO arranged within the virtual referenceframe VRF and/or control the surgical tool 122 to facilitate guidedexecution of one or more intraoperatively-planned surgical steps, as isdescribed in greater detail below.

With continued reference to FIG. 1, the surgical tool 122 is movablerelative to the target site TS and is configured to engage, remove, cut,manipulate, or otherwise effect treatment of tissue during the executionof the surgical procedure. To this end, the illustrated embodiment ofthe surgical tool 122 depicted throughout the drawings is realized as ahand-held rotary instrument comprising an energy applicator 124, anactuator 126, one or more tool control inputs 128, a tool controller130, a tool tracker 132 (depicted schematically in FIG. 1). The energyapplicator 124 is configured to engage tissue at the target site TSduring execution of the surgical procedure, and may be of a number ofdifferent types, styles, and/or configurations (e.g., a drill bit, a sawblade, a bur, a vibrating tip, and the like). The actuator 126 iscoupled to and drives the energy applicator 124 (e.g., by generatingrotational torque), and may likewise be of different types, styles,and/or configurations (e.g., an electric motor, an ultrasonictransducer, and the like). The tool control input 128 is arranged foractuation by the surgeon to facilitate driving the energy applicator124. It will be appreciated that the tool control input 128 could beconfigured in a number of different ways sufficient to be actuated bythe surgeon (e.g., with buttons, triggers, switches, knobs, levers,touchscreens, and the like).

In the illustrated embodiment, the tool control input 128 communicateswith the tool controller 130 which, in turn, communicates with thecomputing device 108, the navigation system 104, and/or other componentsof the surgical system 100, such as via physical electrical connections(e.g., a tethered wire harness) and/or via one or more types of wirelesscommunication (e.g., with a WiFi™ network, Bluetooth®, a radio network,and the like). It will be appreciated that the tool controller 130 maybe configured in a number of different ways and could comprise and/orsupport various types of electrical components (e.g., sensors,processors, integrated circuits, transceivers, and the like) in someembodiments. Examples of the utilization of tool controllers 130 of thistype for execution of surgical procedures can be found in U.S. Pat. No.9,226,796, entitled “Method for Detecting a Disturbance as an EnergyApplicator of a Surgical Instrument Traverses a Cutting Path,” thedisclosure of which is hereby incorporated by reference.

In the representative embodiment illustrated herein, the tool tracker132 is firmly affixed to the surgical tool 122 and enables thenavigation system 104 to track states of the surgical tool 122, asdescribed in greater detail below. In some embodiments, the surgicaltool 122 may define a predetermined tool reference point TRP whichrepresents a relative position of the energy applicator 124 within atool coordinate system TCS defined by the position and orientation ofthe tool tracker 132 relative to energy applicator 124 (see FIG. 18).

In addition to driving the actuator 126 based on communication with thetool control input 128 (e.g., such as in response to signals generatedby the tool control input 128), the tool controller 130 may also drivethe actuator 126 based on communication with other control inputsassociated with the surgical system 100, such as a foot pedal controlinput 134 (see FIG. 1). Furthermore, in some embodiments, and as isdescribed in greater detail below, the tool controller 130 may alsodrive the actuator 126 based on communication with other components ofthe surgical system 100 (e.g., the navigation system 104) in order tofacilitate operating the actuator 126 in different ways based on therelative position and/or orientation of the surgical tool 122 relativeto the target site TS (e.g., to facilitate guided execution of one ormore intraoperatively-planned surgical steps).

As noted above, the surgical tool 122 could be of a number of differenttypes and/or configurations to engage tissue at the target site TSduring execution of the surgical procedure. In some embodiments, thesurgical tool 122 could be configured similar to as is shown in U.S.Patent Application Publication No. 2013/0060278, entitled, “SurgicalInstrument Including Housing, a Cutting Accessory that Extends from theHousing and Actuators that Establish the Position of the CuttingAccessory Relative to the Housing”, the disclosure of which is herebyincorporated by reference.

While the surgical tool 122 illustrated herein is configured forun-tethered, hand-held operation by the surgeon, as noted above, thesurgical tool 122 could also be realized with an end effector attachedto a manipulator of a surgical robot configured to move the energyapplicator 124 relative to the target site TS by articulating themanipulator (not shown). One example arrangement of a surgical tool 122realized with a surgical robot end effector manipulator is described inU.S. Pat. No. 9,119,655, entitled, “Surgical Manipulator Capable ofControlling a Surgical Instrument in Multiple Modes,” the disclosure ofwhich is hereby incorporated by reference. Another example arrangementof a surgical tool 122 realized with a surgical robot end effectormanipulator is described in U.S. Patent Application Publication No.2014/0276949, filed on Mar. 15, 2014, entitled, “End Effector of aSurgical Robotic Manipulator,” the disclosure of which is herebyincorporated by reference.

Referring now to FIGS. 1 and 18-19A, as noted above, the surgical system100 employs the navigation system 104 (sometimes referred to as a“tracking system”) to, among other things, track states of thedigitization device 106 and the surgical tool 122. More specifically,the navigation system 104 is configured to track states of the pointertracker 120 and the tool tracker 132. Furthermore, in some embodiments,the navigation system 104 is also configured to track states of a firstpatient tracker 136, and/or a second patient tracker 138, respectivelyattached to different portions of the patient's anatomy relative to thetarget site TS. In the embodiment depicted in FIG. 1, the first patienttracker 136 is firmly affixed to the femur F of the patient's anatomy,and the second patient tracker 138 is firmly affixed to the tibia T ofthe patient's anatomy. In some embodiments, the first patient tracker136 (and/or the second patient tracker 138) may define a patientcoordinate system PCS (see FIG. 18).

In some embodiments, the surgical system 100 may also comprise ahead-mountable display (HMD) unit 140. As is described in greater detailbelow, the HMD unit 140 is configured to render a visualization ofcertain aspects the CAD program 102 such that the visualization isvisible to the surgeon overlaid onto the target site TS with augmentedreality and/or mixed reality (see FIGS. 19F, 40B, and 46B). Here too,the navigation system 104 is configured to track states of the HMD unit140. To this end, the HMD unit 140 comprises a display tracker 142 whichis firmly affixed to the HMD unit 140. In some embodiments, the displaytracker 142 may define a display coordinate system HCS (see FIG. 18).

As is depicted in FIG. 1, the illustrated navigation system 104 includesa cart assembly 144 that houses a navigation controller 146 which isdisposed in communication with one or more display units 148, one ormore control inputs 150, and a localizer 152, each of which aredescribed in greater detail below. One example of the type of navigationsystem 104 described herein is shown in U.S. Pat. No. 9,008,757, filedon Sep. 24, 2013, entitled, “Navigation System Including Optical andNon-Optical Sensors,” hereby incorporated by reference.

The navigation controller 146 may be of a number of different styles,types, or configurations, and may also be disposed in communication withother components of the surgical system 100, such as the digitizationdevice 106 (e.g., the pointer control input 116 and/or the pointercontroller 118), the surgical tool 122 (e.g., the tool control input 128and/or the tool controller 130), the foot pedal control input 134, thecomputing device 108, and the and the like. In some embodiments, thenavigation controller 146 may comprise the computing device 108. Heretoo, communication between the navigation controller 146 and the variousother components of the surgical system 100 described herein may occurin a number of different ways, including by various types of wiredand/or wireless communication.

The display units 148 of the navigation system 104 may be configured todisplay a navigation interface in operative communication with thenavigation controller 146 (navigation interface not shown) to thesurgeon or another user. It will be appreciated that the one or moredisplay units 148 could be configured in a number of different wayswithout departing from the scope of the present disclosure.

In some embodiments, one or more of the display units 148 are configuredto display aspects of the CAD program 102 to the surgeon. For example,in some embodiments, the CAD program 102 is configured to render avisualization VIZ of the virtual reference frame VRF which may bedisplayed by one or more of the display units 148 of the navigationsystem 104 (see FIGS. 9A-17, 19A-40A, 41-46A, and 47-49), and/or by theHMD unit 140 in some embodiments (see FIGS. 19F, 40B, and 46B).

Furthermore, and as is described in greater detail below, the CADprogram 102 may be further configured to render, within thevisualization VIZ of the virtual reference frame VRF: a virtualrepresentation of the digitization device 106 (hereinafter, “virtualdigitization device 106V”) having a position and/or orientation derivedfrom the tracked states SZ of the digitization device 106 (see FIGS.19A-19D, 20A-40A, and 41-45); a virtual representation of the surgicaltool 122 (hereinafter, “virtual surgical tool 122V”) having a positionand/or orientation derived from the tracked states SZ of the surgicaltool 122 (see FIGS. 46A and 47-49); a virtual representation of thefirst patient tracker 136 (hereinafter, “virtual first patient tracker136V”) having a position and/or orientation derived from the trackedstates SZ of the first patient tracker 136 (see FIGS. 19A-40A, 41-46A,and 47-49); and/or a virtual representation of the second patienttracker 138 (hereinafter, “virtual second patient tracker 138V”) havinga position and/or orientation derived from the tracked states SZ of thesecond patient tracker 138 (see FIGS. 20A-26B).

Furthermore, in some embodiments, the CAD program 102 may be configuredto render, within the visualization VIZ of the virtual reference frameVRF: a virtual representation of the digitizer coordinate system DCS(hereinafter, “virtual digitizer coordinate system DCSV”) associatedwith the virtual digitization device 106V; a virtual representation ofthe patient coordinate system PCS (hereinafter, “virtual patientcoordinate system PCSV”) associated with the virtual first patienttracker 136V; and/or a virtual representation of the tool coordinatesystem TCS (hereinafter, “virtual tool coordinate system TCSV”)associated with the virtual first patient tracker 136V (virtual toolcoordinate system TCSV not shown).

Further still, in some embodiments, the CAD program may be configured torender, within the visualization VIZ of the virtual reference frame VRF:a virtual representation of the digitizer reference point DRP(hereinafter, “virtual digitizer reference point DRPV”) having aposition known relative to the virtual digitizer coordinate system DCSVwithin the virtual reference frame VRF; and/or a virtual representationof the tool reference point TRP (hereinafter, “virtual tool referencepoint TRPV”) having a position known relative to the virtual toolcoordinate system TCSV within the virtual reference frame VRF.

The control inputs 150 of the navigation system 104 are generallyconfigured to facilitate operating the navigation interface (not shown)of the navigation system 104 and, in some embodiments, are alsoconfigured to facilitate operating the CAD program 102. To this end, thecontrol inputs 150 depicted in FIG. 1 comprise interactive touchscreenscoupled to the display units 148. However, the control inputs 150 mayinclude any one or more of a keyboard, a mouse, a microphone (e.g., forvoice-activation), a gesture-based control device, and the like. Inembodiments where the digitization device 106 is configured tofacilitate operating the CAD program 102, it may also serve as one ofthe control inputs 150 of the navigation system 104. For example, thedigitization device 106 may be serve as a control input 150 based ontracked states SZ of the digitization device 106, based on actuation ofthe pointer control input 116, and/or based on signals generated by thepointer controller 118 (e.g., from sensors of various types, includinginertial sensors, accelerometers, gyroscopes, and the like). Otherconfigurations are contemplated.

Referring again to FIGS. 1 and 18-19A, as noted above, the navigationsystem 104 is configured to track states of the pointer tracker 120, thetool tracker 132, the first patient tracker 136, the second patienttracker 138, the display unit tracker 142, and/or other trackersutilized during the surgical procedure. More specifically, the localizer152 is configured to sense, track, or otherwise monitor the positionand/or orientation (the “pose”) of the respective trackers at respectivecoordinates within a localizer coordinate system LCLZ based, forexample, on the relative pose of the digitizer coordinate system DCS,the tool coordinate system TCS, the patient coordinate system PCS,and/or the display coordinate system HCS within the localizer coordinatesystem LCLZ (see FIG. 18).

The localizer 152 monitors the trackers 120, 132, 136, 138, 142 todetermine a state of each of the trackers 120, 132, 136, 138, 142 whichcorresponds to the state of the object respectively attached thereto.The navigation controller 146 gathers data about the tracked states SZof each tracker 120, 132, 136, 138, 142 monitored by the localizer 152within the localizer coordinate system LCLZ. The navigation controller146 communicates the tracked states SZ of one or more of the trackers120, 132, 136, 138, 142 to the computing device 108 implementing the CADprogram 102 (shown in FIG. 2), which can be used to facilitate ad-hocintraoperative planning of surgical steps as noted above and as isdescribed in greater detail below.

As used herein, the term “tracked state SZ” includes, but is not limitedto, data which represents or defines the position and/or orientation ofa tracked object, and/or equivalents or derivatives of the positionand/or orientation. For example, a tracked state SZ may be a pose of thetracked object, and may include linear data, angular velocity data, andthe like. Other configurations are contemplated.

In the illustrated embodiment, the localizer 152 is an optical localizerand includes a camera unit 154. The camera unit 154 has an outer casing156 that houses one or more optical sensors 158. Here, the opticalsensors 158 are configured to sense movement of the various trackers120, 132, 136, 138, 142. To this end, any one or more of the trackers120, 132, 136, 138, 142 may include active markers 160 (see FIG. 18).The active markers 160 may include light emitting diodes (LEDs).Alternatively, the trackers 120, 132, 136, 138, 142 may have passivemarkers, such as reflectors which reflect light emitted from the cameraunit 154 or another predetermined light source. Other suitable markersnot specifically described herein may be utilized.

Although one embodiment of the navigation system 104 illustratedthroughout the drawings, the navigation system 104 may have any othersuitable configuration for tracking the HMD unit 140, the surgical tool122, the digitization device 106, and/or trackers attached to portionsof the patient's anatomy at the target site TS. In some embodiments, thenavigation system 104 and/or the localizer 152 are ultrasound-based. Forexample, the navigation system 104 may comprise an ultrasound imagingdevice coupled to the navigation controller 146 and configured tofacilitate acquiring ultrasound images (e.g., of the HMD unit 140, thesurgical tool 122, the digitization device 106, the trackers attached atthe target site TS, and the like) such that tracked states SZ arecommunicated to (or interpreted by) the navigation controller 146 basedon the ultrasound images. The ultrasound images may be 2D, 3D, or acombination thereof. The navigation controller 146 may processultrasound images in near real-time to determine the tracked states SZ.The ultrasound imaging device may have any suitable configuration andmay be different than the camera unit 154 as shown in FIG. 1.

In some embodiments, the navigation system 104 and/or the localizer 152are radio frequency (RF) based. For example, the navigation system 104may comprise an RF transceiver coupled to the navigation controller 146.The HMD unit 140, the surgical tool 122, the digitization device 106,and the trackers attached to portions of the patient's anatomy at thetarget site TS may comprise RF emitters or transponders attachedthereto. The RF emitters or transponders may be passive, or may beactively energized. The RF transceiver transmits an RF tracking signal,and the RF emitters respond with RF signals such that tracked states SZare communicated to (or interpreted by) the navigation controller 146.The RF signals may be of any suitable frequency. The RF transceiver maybe positioned at any suitable location to track the objects using RFsignals effectively. Furthermore, it will be appreciated thatembodiments of RF-based navigation systems may have structuralconfigurations that are different than the navigation system 104illustrated throughout the drawings.

In some embodiments, the navigation system 104 and/or localizer 152 areelectromagnetically (EM) based. For example, the navigation system 104may comprise an EM transceiver coupled to the navigation controller 146.The HMD unit 140, the surgical tool 122, the digitization device 106,and the trackers attached to portions of the patient's anatomy at thetarget site TS may comprise EM components attached thereto (e.g.,various types of magnetic trackers, electromagnetic trackers, inductivetrackers, and the like) which may be passive, or may be activelyenergized. The EM transceiver generates an EM field, and the EMcomponents respond with EM signals such that tracked states SZ arecommunicated to (or interpreted by) the navigation controller 146. Thenavigation controller 146 may analyze the received EM signals toassociate relative states thereto. Here too, it will be appreciated thatembodiments of EM-based navigation systems may have structuralconfigurations that are different than the navigation system 104illustrated throughout the drawings.

Those having ordinary skill in the art will appreciate that thenavigation system 104 and/or localizer 152 may have any other suitablecomponents or structure not specifically recited herein. Furthermore,any of the techniques, methods, and/or components described above withrespect to the camera-based navigation system 104 shown throughout thedrawings may be implemented or provided for any of the other embodimentsof the navigation system 104 described herein. For example, thenavigation system 104 may utilize solely inertial tracking or anycombination of tracking techniques.

Referring now to FIG. 18, as noted above, the localizer 152 of thenavigation system 104 is able to track states of each of the trackers120, 132, 136, 138, 142 within the localizer coordinate system LCLZ.Thus, the navigation controller 146 is able to simultaneously monitor,within the localizer coordinate system LCLZ, changes in the positionand/or orientation of: the digitizer coordinate system DCS (or theposition of the digitizer reference point DRP relative thereto); thetool coordinate system TCS (or the position of the tool reference pointTRP relative thereto); the patient coordinate system PCS; and/or thedisplay coordinate system HCS. Here, data associated with the pose ofone or more of the trackers 120, 132, 136, 138, 142 within the localizercoordinate system LCLZ may be translated (e.g., with the navigationcontroller 146, with the computing device 108, and the like) into anarbitrary CAD coordinate system CCS within the virtual reference frameVRF of the CAD program 102, and/or vice-versa, using any suitabletransformation techniques. Thus, in some embodiments, each registeredlocal virtual reference LVR has X,Y,Z coordinates within the CADcoordinate system CCS, which may be stored in a database, table, list,and the like (e.g., on the memory device 112). One example of thetranslation or transformation of data between coordinate systems isdescribed in U.S. Pat. No. 8,675,939, entitled “Registration ofAnatomical Data Sets”, the disclosure of which is hereby incorporated byreference in its entirety.

To illustrate the concept of translating between the localizercoordinate system LCLZ and the CAD coordinate system CCS, FIG. 19A showsthe localizer 152, the first patient tracker 136 fixed to the femur Fadjacent the target site TS, and the digitization device 106 arrangedwith the pointer tip 114 at the trochanter minor of the femur F. Here,transforms TR are shown generically between the localizer coordinatesystem LCLZ and the digitizer coordinate system DCS, and between thelocalizer coordinate system LCLZ and the patient coordinate system PCS.

FIG. 19A also shows the corresponding visualization VIZ of the virtualreference frame VRF rendered by the CAD program 102 on one of thedisplay units 148. The visualization VIZ depicted in FIG. 19A shows theCAD coordinate system CCS, the virtual digitization device 106V, and thevirtual first patient tracker 136V. Moreover, the visualization VIZdepicted in FIG. 19A also shows the virtual digitizer coordinate systemDCSV associated with the virtual digitization device 106V, and thevirtual patient coordinate system PCSV associated with the virtual firstpatient tracker 136V. Here, virtual transforms VTR are shown generically(represented with solid lines) between the CAD coordinate system CCS andthe virtual digitizer coordinate system DCSV, and between the CADcoordinate system CCS and the virtual patient coordinate system PCSV.Thus, by tracking states of the trackers 120, 132, 136, 138, 142 withinthe localizer coordinate system LCLZ, transforms TR with respect to thelocalizer coordinate system LCLZ can be translated into related virtualtransforms VTR with respect to the CAD coordinate system CCS, andvice-versa.

Furthermore, because the position of the pointer tip 114 (which definesthe digitizer reference point DRP) is known relative to the pointertracker 120 (which defines the digitizer coordinate system DCS), andbecause the pointer tracker 120 is tracked by the navigation system 104within the localizer coordinate system LCLZ, the navigation controller146 (and/or the computing device 108) can translate coordinates of thedigitizer reference point DRP from the localizer coordinate system LCLZinto corresponding coordinates of a local virtual reference LVR withinthe CAD coordinate system CCS, thereby facilitating registration oflocal virtual references LVR within the virtual reference frame VRF.

Put differently, in order to register a local virtual references LVRwithin the virtual reference frame VRF of the CAD program 102, thesurgeon can activate the pointer control input 116 of the digitizationdevice 106 to establish a coordinate point associated with the positionof the pointer tip 114 within the localizer coordinate system LCLZ(e.g., at a portion of the patient's anatomy at the target site TS).Here, activation of the pointer control input 116 facilitatesdetermining coordinates of the digitizer reference point DRP within thelocalizer coordinate system LCLZ, and those coordinates can betranslated to the CAD coordinate system CCS and registered as a localvirtual reference LVR within the virtual reference frame VRF of the CADprogram 102.

With continued reference to FIGS. 1 and 18-19A, it will be appreciatedthat the CAD coordinate system CCS may be arbitrary and may notnecessarily correspond to the localizer coordinate system LCLZ incertain embodiments. By way of illustrative example, two registeredlocal virtual references LVR within the virtual reference frame VRF maybe positioned arbitrarily with respect to the CAD coordinate system CCS,but may be positioned relative to one another within the CAD coordinatesystem CCS in a way which corresponds to the respective positions thatthe digitizer reference point DRP was at within the localizer coordinatesystem LCLZ when the local virtual references LVR were established.

Furthermore, different registered local virtual references LVR can beassociated with (or transformed dynamically relative to) othercoordinate systems within the virtual reference frame VRF in someembodiments. By way of illustrative example, different local virtualreferences LVR (and/or geometrical design objects GDO arranged relativethereto) may be associated with respectively different parts of thepatient's anatomy relative to the target site TS (e.g., some with thetibia T and some with the femur F) such that relative movement betweenthe different parts of the patient's anatomy results in correspondingrelative movement between the different local virtual references LVR(and/or geometrical design objects GDO arranged relative thereto). Here,it will be appreciated that relative movement between any of thetrackers 120, 132, 136, 138, 142 with respect to the localizercoordinate system LCLZ may be rendered, represented, or otherwiseobserved within the virtual reference frame VRF as correspondingrelative movement with respect to the CAD coordinate system CCS.

Referring now to FIG. 6, various different example geometrical designobjects GDO which can be arranged within the virtual reference frame VRFwith the CAD program 102 are shown. The illustrated geometrical designobjects GDO include a point PT (or “landmark”), a point cloud PC, a lineLN (or “axis”), a line segment LS, a plane PN, a cylinder CY, a wedgeWD, a triangle mesh TM, an osteotomy plane OP, a complex cut CX, achevron cut CV, and a scarf cut CS. As will be appreciated from thesubsequent description below, each of the geometrical design objects GDOintroduced above can be arranged within the virtual reference frame VRFin a number of different ways, and in connection with ad-hocintraoperative planning a number of different types of surgical steps.

As is explained in greater detail below, the CAD program 102 isconfigured to facilitate arranging the various geometrical designobjects GDO within the virtual reference frame VRF based on geometricrelationships with respect to one or more registered local virtualreferences LVR, and/or one or more calculated virtual references CVRderived from one or more registered local virtual references LVR. Eachtype of geometrical design object GDO must be “defined,” “constrained,”or “fixed” by a minimum (and sometimes maximum) number of registeredlocal virtual references LVR and/or calculated virtual references CVR.

For the purposes of clarity and consistency, in order to illustrate theconcept of “defining” geometrical design objects GDO, the term “virtualreference(s)” will occasionally be used herein to refer to eitherregistered local virtual references LVR or calculated virtual referencesCVR. Here, a “virtual reference” may be considered as a set of X,Y,Zcoordinates within the virtual reference frame VRF.

Points PT are the most primitive type of geometrical design object GDOand are defined with (and are synonymous to) a single virtual reference.Thus, whenever a registered local virtual reference LVR is establishedusing the digitization device 106, and/or whenever a calculated virtualreference CVR is established (e.g., derived from one or more registeredlocal virtual references LVR), a point PT may be created and serve as ageometrical design object GDO arranged within the virtual referenceframe VRF. In the embodiments illustrated throughout the drawings anddescribed herein, the term “registered local virtual reference LVR” isgenerally used to described a points PT which has been established usingthe digitization device 106. Furthermore, as used herein, the term“landmark” generally refers to a specific portion of the patient'sanatomy at or adjacent to the target site TS which is associated with aspecific point PT.

Lines LN and line segments LS are each defined with two virtualreferences. As illustrated herein, line segments LS comprise a lengthextending between a pair of points PT, whereas lines LN may have aninfinite length (and may also be referred to as “axes”).

In general terms, planes PN are defined with three virtual references.However, and as is described in greater detail below, the CAD program102 of the present disclosure utilizes planes PN as two-dimensionalpolygons that have three or more sides symmetrically arranged at a spanlength SL about an object index OI. The object index OI of planes PN(and certain other geometrical design objects GDO) is analogous to acoordinate system within the virtual reference frame VRF (see, forexample, FIGS. 9A-17). As will be appreciated from the subsequentdescription below, this configuration affords the ability to quicklydefine additional geometrical design objects GDO from planes PN, such asusing a corner of the plane PN as a point PT (or as a calculated virtualreference CVR), using an edge of the plane PN as a line segment LS (oras two calculated virtual references CVR), and the like.

As is described in greater detail below, in some embodiments, the CADprogram 102 is configured to enable construction of compound objectsfrom one or more geometrical design objects GDO arranged within thevirtual reference frame VRF. For example, and with continued referenceto FIG. 6, point clouds PC are comprised of multiple points PT, each ofwhich may be defined by a single virtual reference. Cylinders CY arevolumes which, in general terms, are defined based on the specificgeometric properties of the volume (e.g., selecting a diameter with theCAD program 102) relative to multiple virtual references. Wedges WD maybe represented by two square-shaped planes PN which share a common edgeand may open symmetrically or asymmetrically, or may be constructed ascompound objects such as from two planes PN of different shapes.Triangle meshes TM are compound objects realized as surfaces defined bymultiple points PN, such as from a point cloud PC. Osteotomy planes OPare compound objects and are defined by a pair of parallel planes PNspaced from each other at a cut distance. Complex cuts CX, chevron cutsCV, and scarf cuts CS are each compound objects and are generallydefined by parallel sets of multiple planes PN which share common edges.

It will be appreciated that the geometrical design objects GDO describedherein and illustrated throughout the drawings are non-limiting, andother different geometrical design objects GDO may be utilized incertain embodiments. By way of non-limiting example, geometrical designobjects GDO may be realized as assemblies of one or morepreviously-generated geometrical design objects GDO which are utilizedin a subsequent surgical procedure (e.g., the CAD program 102 mayfacilitate saving, importing, exporting, and the like). Otherconfigurations are contemplated.

Referring now to FIG. 2, as noted above, the CAD program 102 is storedon the memory device 112 of the computing device 108 and is executed bythe processor 110. The processor 110 may include a microprocessor, amicrocontroller, or the like. The memory device 112 is a non-transitorycomputer-readable storage medium that stores computer-readable andexecutable instructions embodied in one or more programs or modules. Thememory device 112 may include, for example, non-volatile memory such asa hard disk or flash memory, and may also include random access memory(RAM), which can include non-volatile RAM (NVRAM), magnetic RAM (MRAM),ferroelectric RAM (FeRAM), or any other suitable memory.

In some embodiments, the computing device 108 is configured tocommunicate with one or more of the display units 148, the navigationcontroller 146, the tool controller 130, and/or the pointer controller118. As noted above, in some embodiments, the computing device 108 isrealized with the navigation controller 146 or is incorporated into thenavigation controller 146. In another embodiment, the computing device108 is a separate computer, device, and the like. In yet anotherembodiment, the computing device 108 forms part of a portable electronicdevice 162 (e.g., a tablet computer). Other configurations arecontemplated.

Referring now to FIG. 3, an example software architecture of the CADprogram 102 is shown with various modules employed to facilitateplanning of surgical steps and execution of the surgical procedure, asnoted above and as described in greater detail below. These modulesinclude a point cloud toolkit 164, a medical imaging toolkit 166, amathematical toolkit 168, a physics toolkit 170, a graphical userinterface (GUI) 172, and a module for CAD functions and algorithms 174(hereinafter “algorithm module”). Also shown in FIG. 3 arerepresentations of visualization data VZ which may be available duringplanning of surgical steps, and tool control data CZ which may beutilized during execution of the surgical procedure, both of which willbe described in greater detail below.

The point cloud toolkit 164 is employed in certain embodiments tofacilitate mapping point clouds PC to anatomical structures or othergeometrical design objects GDO. The point cloud toolkit 164 may includealgorithms that enable the CAD program 102 to generate triangle meshesTM or other surfaces, and/or to define a point cloud PC generated usingthe digitization device 106 relative to an appropriate geometricaldesign object GDO arranged in the virtual reference frame VRF. Otherconfigurations are contemplated.

The medical imaging toolkit 166 comprises a set of software tools which,among other things: enable the segmentation, registration, and displayof portions of the patient's anatomy; enable 3D graphics processing,image processing, and visualization; provide an application framework;facilitate operation of the GUI 172; and facilitate implementation ofthe algorithm module 174. In one embodiment, the medical imaging toolkit166 comprises or is based on one or more open-source frameworks known inthe art, such as the Medical Imaging Interaction Toolkit (MITK), theInsight Segmentation and Registration Toolkit (ITK), and/or theVisualization Toolkit (VTK). Other configurations are contemplated.

The mathematical toolkit 168 may include a linear algebra library thatmay be utilized for pattern recognition, signal processing, and thelike. In some embodiments, the mathematical toolkit 168 may facilitatearranging or moving geometrical design objects GDO within the virtualreference frame VRF (e.g., rotation, translation, and the like). In oneembodiment, the mathematical toolkit 168 comprises or is based on one ormore open-source C++ libraries known in the art, such as Armadillo,and/or Boost. Other configurations are contemplated.

The physics toolkit 170 may include a library that helps facilitatesimulation of and/or execution of the surgical procedure (e.g., based oncollision detection, rigid body dynamics, and the like), such as may beused to define virtual boundaries with geometrical design objects GDOarranged within the virtual reference frame VRF that are used togenerate tool control data CZ communicated to and interpreted by thetool controller 130 of the surgical tool 122, as noted above and as isdescribed in greater detail below in connection with FIGS. 27-49. In oneembodiment, the physics toolkit 170 comprises or is based on one or moreopen-source physics engines known in the art, such as Bullet. Otherconfigurations are contemplated.

In one embodiment, the GUI 172, which is described in greater detailbelow in connection with FIGS. 5, 7, and 20A-26B, comprises or is basedon an open-source application framework such as Qt and/or the QtModeling Language (QML). Other configurations are contemplated.

Referring now to FIG. 8, various example algorithms of the algorithmmodule 174 are shown which can be used to facilitate arranginggeometrical design objects GDO within the virtual reference frame VRFwith the CAD program 102. In some embodiments, the algorithm module 174comprises or is based on an open-source framework such as theVisualization Toolkit (VTK). Other configurations are contemplated. Inthe subsequent description of the example algorithms below, the term“pointer tip” will briefly be used herein to refer to the relativeposition of the digitizer reference point DRP within the virtualreference frame VRF relative to certain geometrical design objects GDO.

A first algorithm AL01 is configured to facilitate arranging a point PTat a center of mass calculated using a plurality of points PT or a pointcloud PC.

A second algorithm AL02 is configured to facilitate arranging a point PTat the intersection of a plurality of lines LN and/or line segments LS.

A third algorithm AL03 is configured to facilitate arranging a point PTat the intersection of a plurality of planes PN.

A fourth algorithm AL04 is configured to facilitate arranging a point PTat the intersection of plane PN and a line LN or line segment LS.

A fifth algorithm AL05 is configured to facilitate arranging a point PTalong a line LN or a line segment LS based on the closest distance tothe pointer tip (e.g., projecting a point PT onto a line LN and movingit along the line LN based on movement of the digitation device 106).

A sixth algorithm AL06 is configured to facilitate arranging a point PTalong a plane PN based on the closest distance to the pointer tip (e.g.,projecting a point PT onto a plane PN and moving it about the plane PNbased on movement of the digitation device 106).

A seventh algorithm AL07 is configured to facilitate arranging a pointPT at a percentage of distance between: the pointer tip; and a localvirtual reference LVR, or a calculated virtual reference CVR (e.g.,positioning a point PT halfway between the pointer tip and a registeredlocal virtual reference LVR).

An eighth algorithm AL08 is configured to facilitate arranging a pointPT at a percentage of distance along a line segment LS (e.g.,positioning a point PT halfway between two registered local virtualreferences LVR used to construct or define the line segment LS).

A ninth algorithm AL09 is configured to facilitate arranging a line LNor a line segment LS at the intersection of two planes PN.

A tenth algorithm AL10 is configured to facilitate arranging a line LNprojected onto a plane PN (e.g., projecting a line LN onto a plane PNand moving it about the plane PN based on movement of the digitationdevice 106).

An eleventh algorithm AL11 is configured to facilitate arranging a lineLN through a point PT which is parallel to another line LN within thevirtual reference frame VRF.

A twelfth algorithm AL12 is configured to facilitate arranging a linesegment LS between two points PT.

A thirteenth algorithm AL13 is configured to facilitate arranging aplane PN from three points PT.

A fourteenth algorithm AL14 is configured to facilitate arranging aplane PN from a point PT and a line LN or a line segment LS (e.g., witha normal vector of the plane PN arranged perpendicular to the line LNand with the object index OI of the plane PN positioned at the pointPT).

A fifteenth algorithm AL15 is configured to facilitate arranging a planePN from two parallel lines LN and/or line segments LS (e.g., with theplane PN arranged coincident with both lines LN).

A sixteenth algorithm AL16 is configured to facilitate arranging a planePN through a point PT which is parallel to another plane PN within thevirtual reference frame VRF.

A seventeenth algorithm AL17 is configured to facilitate aligning aplane PN normal to a point PT (e.g., with a normal vector of the planePN intersecting the point PT).

An eighteenth algorithm AL18 is configured to facilitate arranging aplane PN through a plurality of points PT or a point cloud PC (e.g., bycalculating a regression plane through a point cloud PC).

A nineteenth algorithm AL19 is configured to facilitate arranging anosteotomy plane OP from a plane PN (e.g., by creating a plane PN whichis parallel to and is spaced from another plane PN within the virtualreference frame VRF at a cut distance).

A twentieth algorithm AL20 is configured to facilitate arranging a wedgeWD from a line LN and a plane PN (e.g., by creating a plane PN which iscoincident with a line LN within the virtual reference frame VRF andwhich intersects another plane PN within the virtual reference frameVRF).

A twenty-first algorithm AL21 is configured to facilitate arranging awedge WD from a plane PN (e.g., by creating a plane PN which shares anedge with another plane PN within the virtual reference frame VRF).

A twenty-second algorithm AL22 is configured to facilitate arranging acylinder CY from a plurality of points PT or a point cloud PC (e.g., bycalculating a line LN through a point cloud PC and arranging a cylinderCY concentrically to the line LN).

A twenty-third algorithm AL23 is configured to facilitate arranging acylinder CY from a line LN or a line segment LS (e.g., by arranging acylinder CY concentrically to the line LN or line segment LS).

A twenty-fourth algorithm AL24 is configured to facilitate defining asurface from a plurality of points PT or a point cloud PC (e.g., bycreating triangle mesh TM from a point cloud PC).

A twenty-fifth algorithm AL25 is configured to facilitate mergingmultiple surfaces (e.g., to construct a volume using a triangle mesh TMand one or more planes PN).

A twenty-sixth algorithm AL26 is configured to facilitate merging pointsPT (e.g., merging two point clouds PC into a single point cloud PC, ormerging a point cloud PC with additional points PT).

A twenty-seventh algorithm AL27 is configured to facilitate clipping asurface with one or more planes PN (e.g., removing points PT of atriangle mesh TM above or below a plane PN which intersects the trianglemesh TM).

A twenty-eighth algorithm AL28 is configured to facilitate cutting awedge WD out from surfaces (e.g., removing points PT of a triangle meshTM above or below planes PN which share a common edge to form a wedgeWD).

A twenty-ninth algorithm AL29 is configured to facilitate cutting awedge WD out from a point cloud PC (e.g., removing points PT of a pointcloud PC above or below planes PN which share a common edge to form awedge WD).

A thirtieth algorithm AL30 is configured to facilitate cutting a volumefrom a surface (e.g., removing points PT of a triangle mesh TM relativeto a cylinder CY which intersects the triangle mesh TM).

A thirty-first algorithm AL31 is configured to facilitate creatingcomplex cuts CX from planes PN (e.g., creating a set of multiple planesPN which share common edges where the set is parallel to and spaced fromanother set of multiple planes PN which share common edges).

A thirty-second algorithm AL32 is configured to facilitate creatingcomplex cuts CX from line segments LS (e.g., creating a set of multipleplanes PN which share a common edge defined by a line segment LS andcreating another set of multiple planes PN which is parallel to andspaced from the first set).

It will be appreciated that the algorithms described above arenon-limiting examples of the functionality of the algorithm module 174of the CAD program 102.

Referring now to FIG. 5, as noted above, the CAD program 102 comprisesthe GUI 172 in the illustrated embodiment, which may be presented to thesurgeon on the one or more display units 148 and may be navigated usingthe one or more control inputs 150 and/or the digitization device 106.The GUI 172 generally comprises a visualization window 176, a viewmanager 178, an object/data manager 180, a new object arrangementsection 182, and an existing object arrangement section 184, each ofwhich will be described in greater detail below in connection with FIGS.20A-26B.

Referring now to FIG. 7, aspects of the new object arrangement section182 and the existing object arrangement section 184 are shown. In thenew object arrangement section 182, three general approaches that can beutilized in arranging new geometrical design objects GDO in order tofacilitate planning the surgical step are represented: a bottom-upapproach 186, a top-down approach 188, and a mixed approach 190. Theseapproaches 186, 188, 190 generally described the different workflowswhich may be utilized for any given surgical procedure. Put differently,the approaches 186, 188, 190 describe different sequences in which newgeometrical design objects GDO can generally be created and arrangedwithin the virtual reference frame VRF.

By way of brief, illustrative example, a bottom-up approach 186 mayinvolve the surgeon selecting an option to create new points PT byestablishing registered local virtual references LVR, selecting anoption to create a new line segment LS using the points PT, andselecting an option to create a new osteotomy plane OP using the linesegment LS. Conversely and also by way of brief, illustrative example, atop-down approach 188 may involve the surgeon selecting an option tocreate a new osteotomy plane OP without having yet established any localvirtual references LVR. Here, the GUI 172 could present the surgeon withdifferent ways to arrange the new osteotomy plane OP, such as with anoption to create the new osteotomy plane OP from a line segment LS. Ifthis option were selected, the GUI 172 could prompt the surgeon toestablish and register the requisite local virtual references LVR neededto define the line segment LS.

A mixed approach 190 is a hybrid of the bottom-up approach 186 and thetop-down approach 188, and involves the GUI 172 providing the surgeonwith contextual options for creating new geometrical design objects GDObased on the types of different geometrical design objects GDO whichhave already been arranged within the virtual reference frame VRF and/orthe based on the presence of registered local virtual references LVRwhich have already been established using the digitization device 106.

It will be appreciated that any one of the approaches 186, 188, 190 maybe utilized for a particular surgical procedure, and may be defined orimplemented in certain embodiments in ways which guide the surgeonthrough a particular surgical procedure. To this end, specificworkflows, macros, and the like could be employed by the CAD program 102to prompt the surgeon to arrange geometrical design objects GDOsequentially in a predetermined order and/or with a contextual surgicalmeaning.

For example, the CAD program 102 could employ a macro forintraoperatively planning a femoral neck osteotomy with a bottom-upapproach 186 and could sequentially: prompt the surgeon to establish alocal virtual reference LVR at a saddle point landmark LM_SP of thefemur F, subsequently prompt the surgeon to establish a local virtualreference LVR at a trochanter minor landmark LM_TM of the femur F, andthen automatically arrange a line segment LS between the registeredlocal virtual references LVR established at the saddle point landmarkLM_SP and the trochanter minor landmark LM_TM, and so on. It will beappreciated that the forgoing example is an illustrative and incompletedescription of initial surgical steps which may be associated withperforming a femoral neck osteotomy with a bottom-up approach 186 andwith contextual surgical meaning (e.g., prompting the surgeon to performa step with respect to a specific part of the patient's anatomy relativeto the target site TS). Additional examples of each type of approach186, 188, 190 are described below in connection with FIGS. 20A-26B.

With continued reference to FIG. 7, as noted above, aspects of the newobject arrangement section 182 and the existing object arrangementsection 184 are shown. In the new object arrangement section 182, twogeneral modes that can be utilized in creating new geometrical designobjects GDO in order to facilitate planning the surgical step arerepresented: a digitize new references mode 192, and a use existingreferences mode 194. These modes 192, 194 represent the different waysin which new geometrical design objects GDO can be created and/orarranged within the virtual reference frame VRF, and may each beutilized with any of the approaches 186, 188, 190 introduced above.

The digitize new references mode 192 is utilized where the geometricaldesign object GDO is constructed using registered local virtualreferences LVR which are sequentially-established with the digitizationdevice 106 without reliance on previously-established local virtualreferences LVR. For example, a line segment LS may be constructed usingthe digitize new references mode 192 by the surgeon selecting an optionto create the line segment LS and (using a top-down approach 188) theCAD program 102 would prompt the surgeon to sequentially establish twonew local virtual references LVR (each represented as a point PT) which,once registered, then define the line segment LS.

The use existing references mode 194 is utilized where the geometricaldesign object GDO is constructed from (or based on) apreviously-constructed geometrical design object GDO arranged within thevirtual reference frame VRF. Here, one or more local virtual referencesLVR associated with the previously-constructed geometrical design objectGDO are shared with the new geometrical design object GDO. In someembodiments, the new geometrical design object GDO may share one or morelocal virtual references LVR with one or more previously-constructedgeometrical design objects GDO, and the surgeon could use thedigitization device 106 to establish and register one or more additionallocal virtual references LVR to fully define the new geometrical designobject GDO. In other embodiments, the new geometrical design object GDOmay share one or more local virtual references LVR with one or morepreviously-constructed geometrical design objects GDO, and the surgeonwould then use the CAD program 102 to derive (e.g., using the algorithmsmodule 174) and register one or more calculated virtual references CVRto fully define the new geometrical design object GDO. It will beappreciated that the forgoing examples are illustrative andnon-limiting.

With continued reference to FIG. 7, in the new object arrangementsection 182, two general types of new geometrical design objects thatcan be constructed with the CAD program 102 in order to facilitateplanning the surgical step are represented: a primitive type object 196,and a compound type object 198. In general, primitive type objects 196are geometrical design objects GDO which may be used to constructcompound type objects 198.

In some embodiments, primitive type objects 196 comprise points, lines,planes, and volumes which are utilized as “construction objects” inorder to assemble compound type objects 198. In some embodiments,compound type objects 198 may be used to assemble other compound typeobjects 198. It will be appreciated that these descriptions are utilizedherein for illustrative purposes, and that certain types of geometricaldesign objects GDO could be realized as either primitive type objects196 or as compound type objects 198 in certain embodiments.

For example, a wedge WD could be realized as a primitive type object 196that is selectable for construction using a top-down approach 188 wherethe CAD program 102 guides the surgeon through either of the modes 192,194 described above to fully define the wedge WD. Alternatively, a wedgeWD could be realized as a compound type object 198 that is constructedusing a bottom-up approach 186 where the surgeon fully defines one planePN and then uses one of its edges to define another plane PN, where bothfully-defined planes PN are assembled to define the wedge WD as acompound type object 198. Here too it will be appreciated that theforgoing examples are illustrative and non-limiting.

With continued reference to FIG. 7, as noted above, aspects of the newobject arrangement section 182 and the existing object arrangementsection 184 are shown. In the existing object arrangement section 184,two general types of measurements that can be utilized in connectionwith existing geometrical design objects GDO in order to facilitateplanning the surgical step are represented: angle measurements 200 anddistance measurements 202. Here, the CAD program 102 is configured tofacilitate measuring geometric relationships within the virtualreference frame VRF between: geometrical design objects GDO; registeredlocal virtual references LVR; calculated virtual references CVR; and/orother contents of the virtual reference frame VRF (e.g., with respect todifferent transformations or coordinate systems, virtual representationsof tools, trackers, pointers, and the like). In some embodiments, anglemeasurements 200 and/or distance measurements 202 may be performed usingaspects of the mathematical toolkit 168.

In the existing object arrangement section 184, two general options thatcan be utilized in moving existing geometrical design objects GDO withinthe virtual reference frame VRF in order to facilitate planning thesurgical step are represented: a relative to self option 204, and arelative to other object option 206.

The relative to self option 204 may be used to adjust the positionand/or orientation of an existing geometrical design object GDO withinthe virtual reference frame VRF based on its current pose. By way ofillustrative example, the relative to self option 204 may be used torotate or translate a geometrical design object GDO about or withrespect to its object index OI in one or more degrees of freedom (seeFIGS. 10A-10B). In some embodiments, moving a geometrical design objectGDO with the relative to self option 204 may create an additionalgeometrical design object GDO, for example by hiding a pre-movementgeometrical design object GDO and utilizing the local virtual referencesLVR associated with the hidden pre-movement geometrical design object ascalculated virtual references CVR for the post-movement geometricaldesign object GDO. In this illustrative example, the plane PN depictedin FIG. 10A represents a pre-movement geometrical design object GDO, andthe plane PN depicted in FIG. 10B represents a post-movement geometricaldesign object GDO.

The relative to other object option 206 may be used to adjust theposition and/or orientation of an existing geometrical design object GDOwithin the virtual reference frame based geometrical relationships, suchas those described above in connection with angle measurements 200and/or distance measurements 202. Moreover, existing geometrical designobjects GDO can be “constrained” or positioned in particular waysrelative to other geometrical design objects GDO, registered localvirtual references LVR, and/or calculated virtual references CVR. Someof these constraints and/or geometrical relationships could be based onaspects of or carried out using the algorithms module 174 describedabove in connection with FIG. 8, and/or the mathematical toolkit 168.

To illustrate examples of constraints, FIG. 12 depicts an octagonalplane PN shown with its object index OI constrained to a point PT of aline segment LS. If desired, the surgeon could rotate the plane PN aboutits object index OI. By way of further example, FIG. 13 shows the sameline segment LS and octagonal plane PN, but with the object index OI ofthe plane PN positioned midway along the line segment LS and with theobject index OI orientated such that the octagonal plane PN (e.g., itsnormal vector) is perpendicular to the line segment LS. In someembodiments, the arrangement depicted in FIG. 13 may be achieved usingthe eighth algorithm AL08 to create a calculated virtual reference CVRrealized as a point PT midway along the line segment LS, and theseventeenth algorithm AL17 to align the normal vector of the plane PNthrough the point PT realized with the calculated virtual reference CVR.

FIG. 14 depicts an example where two octagonal planes PN are arrangedparallel to and spaced from each other (e.g., to construct an osteotomyplane OP). In some embodiments, the arrangement depicted in FIG. 14 maybe achieved using the nineteenth algorithm AL19 to create one plane PNspaced from and parallel to another plane PN at a cut distance.

FIG. 15 depicts an example where three points PT are projected onto anoctagonal plane PN. In some embodiments, the arrangement depicted inFIG. 15 may be achieved using the sixth algorithm AL06 to project pointsPT on the plane PN.

FIG. 16 depicts an example where a line LN is shown passing through (or“coincident with”) an octagonal plane PN. In some embodiments, thearrangement depicted in FIG. 15 may be achieved using the tenthalgorithm AL10 to project the line LN onto the plane PN.

FIG. 17 depicts a geometrical design objects GDO created by cutting anoctagonal plane PN with a line LN as arranged in FIG. 16. Any one of therepresentative examples described above could be implemented indifferent ways to facilitate moving, constraining, orientating,positioning, or otherwise arranging existing geometrical design objectsGDO using the relative to other object option 206, based such as onaspects of the algorithms module 174 (see FIG. 8) and/or aspects of themathematical toolkit 168. Other configurations are contemplated.

Referring again to FIG. 7, in the existing object arrangement section184, four general types of adjustments that can be utilized inconnection with existing geometrical design objects GDO in order tofacilitate planning the surgical step are represented: visibilityadjustment 208, geometrical parameters adjustments 210, trackerassignment adjustments 212, and definition adjustments. As is describedin greater detail below in connection with FIGS. 20A-26B, the visibilityadjustments 208 can be used to change how geometrical design objects GDOare rendered in the visualization VIZ of the virtual reference frameVRF, such as by changing their color (e.g., to differentiate from othergeometrical design objects GDO), applying and/or editing names (e.g., tocorrespond to portions of the anatomy), hiding or showing them (e.g.,hiding one or more primitive type objects 196 used to assemble acompound type object 198), showing dimensions of or associated with thegeometrical design object GDO (e.g., using angle measurements 200 and/ordistance measurements 202), and the like. In some embodiments, certainfeatures afforded by the visibility adjustments 208 can be accessedusing the object/data manager 180, as described in greater detail below.

The geometrical parameters adjustments 210 can be used to facilitatealtering various aspects of certain types of geometrical design objectsGDO; in particular, altering planes PN which, as noted above, areutilized in the CAD program 102 of the present disclosure astwo-dimensional polygons that have three or more sides symmetricallyarranged at the span length SL about their object index OI, which isanalogous to a coordinate system within the virtual reference frame VRF.The geometrical parameters adjustments 210 can be utilized in certainembodiments to, among other things, adjust the span length SL and thenumber of sides of planes PN. As is depicted in FIGS. 9A-9B, the numberof sides of planes PN can be adjusted without altering the pose of theobject index OI within the CAD coordinate system CCS, and withoutaltering the span length SL (four sides shown in FIG. 9A and eight sidesshown in FIG. 9B). Furthermore, as is depicted in FIGS. 11A-11B, thespan length SL of planes PN can be adjusted without altering the pose ofthe object index OI within the CAD coordinate system CCS, and withoutaltering the number of sides (smaller span length SL shown in FIG. 11Aand larger span length SL shown in FIG. 11B). It will be appreciatedthat the ability to alter existing geometrical design objects GDO viathe geometrical parameters adjustments 210 allows certain local virtualreferences LVR associated with those geometrical design objects GDO tobe maintained even where the shape or orientation is subsequentlyaltered.

The tracker assignment adjustments 212 may be used to fix (or“constrain”) geometrical design objects GDO relative to one or more ofthe trackers 120, 132, 136, 138, 142, in one or more degrees of freedom.As is described in greater detail below, fixing geometrical deignobjects GDO, local virtual references LVR, and/or calculated virtualreferences CVR may advantageously allow arrangement of geometricaldesign objects GDO within the virtual reference frame VRF in dynamicways which may be depicted in near-real time with the visualization VIZof the virtual reference frame VRF. Similarly, and as is described ingreater detail below, the CAD program 102 may allow the surgeon to fixcertain geometrical design objects GDO relative to the virtual digitizerreference point DRPV during arrangement of new or existing geometricaldesign objects GDO, such as may be used to orientate the visualizationVIZ relative to the target site TS, and or to orientate the target siteTS relative to the geometrical design objects GDO, local virtualreferences LVR, and/or calculated virtual references CVR rendered withinthe visualization VIZ. Other configuration are contemplated.

In some embodiments, the definition adjustments 214 may be used todefine one or more geometrical design objects GDO arranged within thevirtual reference frame VRF as virtual boundaries 216 (or “no-flyzones,”) which should be avoided by the energy applicator 124 of thesurgical tool 122 during the execution of the surgical procedure. Tothis end, virtual boundaries 216 may be defined with any suitablearrangement, assembly, or combination of geometrical design objects GDOwhere their local virtual references LVR have been fixed (or“constrained”) to one of the patient trackers 136, 138. In someembodiments, virtual boundaries 216 may be defined using an arrangementof geometrical design objects GDO which define a volume or another solidmodel (e.g., a cylinder CY used to define a drill hole with a diameterand a depth along a trajectory). Put differently, one or more virtualboundaries 216 may be used to define or may otherwise be realized as“milling volumes MV” which represent specific portions of the targetsite TS to be removed with the surgical tool 122. In other someembodiments, virtual boundaries 216 may comprise an area of a surfaceand a normal vector associated with the surface (e.g., with a plane PN),or by areas and normal vectors associated with multiple surfaces (e.g.,with a wedge WD, a triangle mesh TM, an osteotomy plane OP, a complexcut CX, a chevron cut CV, a scarf cut CS, and the like). To this end,the CAD program 102 may generate tool control data CZ based ongeometrical design objects GDO which are defined as virtual boundaries216 and/or based on a milling volume MV constructed therefrom (e.g.,based on surface areas, normal vectors to indicate direction, and thelike).

The tool control data CZ may be communicated to and interpreted by thetool controller 130 of the surgical tool 122 (see FIG. 4) for use duringexecution of the surgical procedure to control movement of and/oroperation of the surgical tool 122 relative to the target site TS, suchas by preventing the energy applicator 124 from contacting (orapproaching too close to) a virtual boundary 216 defined with the CADprogram 102, or by slowing or stopping movement of the energy applicator124 as a virtual boundary 216 is contacted (or approached). Here,because the navigation system 104 is able to track states of each oftrackers 120, 132, 136, 138, 142, and because the geometrical designobjects GDO and local virtual references LVR used to define the virtualboundaries 216 (or the milling volume MV) and generate the tool controldata CZ were established within the virtual reference frame VRF as beingfixed to one of the patient trackers 136, 138, then execution of thesurgical procedure with the surgical tool 122 can be navigated and/orguided using the navigation system 104 based on tracked states SZ of thetool tracker 132 and one or more of the patient trackers 136, 138.

Referring now to FIGS. 1, 4, and 7, in some embodiments, the definitionadjustments 214 (see FIG. 7) of the CAD program 102 may be used todefine one or more geometrical design objects GDO arranged within thevirtual reference frame VRF as a virtual implant model 218. Putdifferently, in some embodiments, the CAD program 102 is configured toenable constructing virtual implant models 218 from one or moregeometrical design objects GDO arranged within the virtual referenceframe VRF. Here, in some embodiments, the surgical system 100 mayfurther comprise an implant manufacturing device 220 coupled to thecomputing device 108 which is configured to intraoperatively generate animplant 222 (see FIG. 1) based on the virtual implant model 218, such asusing one or more additive manufacturing techniques. The implant 222 maybe manufactured using one or more materials which are generally inert,are biologically-compatible, which facilitate bone redemonstration, andthe like. Other configurations are contemplated.

Referring now to FIGS. 19A-19F, as noted above, CAD program 102 isconfigured such that the surgeon can modify and/or establishrelationships of local virtual references LVR and/or geometrical designobjects GDO with different patient trackers 136, 138 using, for example,the tracker assignment adjustments 212 (see FIG. 7). In FIG. 19A, forexample, the visualization VIZ of the virtual reference frame VRFdepicts a registered local virtual reference LVR previously establishedat the saddle point landmark LM_SP of the femur F which is spaced fromthe virtual digitization device 106V. Here, the registered local virtualreference LVR of the saddle point landmark LM_SP is fixed (or“constrained”) to the virtual first patient tracker 136V, illustrated bya dash-dash line extending to the virtual patient coordinate systemPCSV.

In FIG. 19B, which depicts another registered local virtual referenceLVR subsequently established at the trochanter minor landmark LM_TM ofthe femur F, the registered local virtual reference LVR of the saddlepoint landmark LM_SP remains fixed to the virtual first patient tracker136V. Here too in FIG. 19B, the registered local virtual reference LVRof the trochanter minor landmark LM_TM is fixed to the virtual firstpatient tracker 136V, likewise illustrated by a dash-dash line extendingto the virtual patient coordinate system PCSV. Moreover, in FIG. 19B(and also in FIGS. 19A and 19C), a dash-dash line extends between thevirtual tool reference point TRPV and the virtual digitizer coordinatesystem DCSV to demonstrate that the virtual tool reference point TRPVmoves concurrently within the virtual reference frame VRF with thevirtual digitization device 106V in response to corresponding movementof the digitization device 106. Movement of this type may be observed bysequentially comparing FIG. 19B, which shows the digitizer referencepoint DRP at the trochanter minor of the femur F, to FIG. 19C, whichshows the digitizer reference point DRP moved away from the femur F.

In FIG. 19D, the visualization VIZ of the virtual reference frame VRFshows an octagonal plane PN which shares an edge with a line segment LSdefined with two established local virtual references LVR which arefixed to the virtual first patient tracker 136V. Here, the object indexOI of the octagonal plane PN has just been defined with another localvirtual reference LVR established based on the illustrated position ofthe virtual digitizer reference point DRPV.

In FIG. 19E, the visualization VIZ of the virtual reference frame VRFshows two octagonal planes PN which share a common edge (along the sameline segment LS described above and illustrated in FIG. 19D), and whichis spaced from the first octagonal plane PN at an angle measurement 200to define a wedge WD. In this illustrative example, the wedge WD isfixed to the virtual first patient tracker 136V based on one or more ofits local virtual references LVR having been established with thedigitization device D relative to the femur F to which the first patienttracker 136 is attached, as noted above.

In FIG. 19F, the same wedge WD illustrated in FIG. 19E is shown, alongwith the registered local virtual references LVR of the saddle pointlandmark LM_SP and the trochanter minor landmark LM_TM, are shown in thevisualization VIZ of the virtual reference frame VRF depicted with thedisplay unit 148. However, in FIG. 19F, the femur F of the patient'sanatomy at the target site TS, and the first patient tracker 136attached to the femur F, are depicted as viewed through the HMD unit 140(see FIGS. 1 and 18) such that the wedge WD and the registered localvirtual references LVR of the saddle point landmark LM_SP and thetrochanter minor landmark LM_TM, are rendered overlaid onto the femur Fwith augmented reality (or “mixed reality”) according to embodiments ofthe present disclosure. Here, because the navigation system 104 is ableto track states of each of trackers 120, 132, 136, 138, 142 and becausethe geometrical design objects GDO and local virtual references LVR wereestablished within the virtual reference frame VRF as being fixed to thefirst patient tracker 136, then the contents of the virtual referenceframe VRF can be presented to the surgeon in mixed reality and candynamically change to compensate for movement of the display unittracker 142 relative to the localizer 152 as the surgeon observes thetarget site TS. In some embodiments, the HMD unit 140 may be providedwith sensors (e.g., inertial sensors, eye-tracking sensors, and thelike) which help facilitate rendering the visualization VIZ overlaidonto the patient's anatomy at the target site TS. Other configurationsare contemplated.

Referring now to FIGS. 20A-20B, a representative embodiment of the GUI172 of the CAD program 102 is shown. In this embodiment (and also in theembodiments illustrated in FIGS. 21A-26B) the GUI 172 employs thevisualization window 176 to render the visualization VIZ of the virtualreference frame VRF to the surgeon. Here too, the CAD program 102 may beconfigured to render, within the visualization VIZ, the variousgeometrical design objects GDO arranged within the virtual referenceframe VRF, registered local virtual references LVR, calculated virtualreferences CVR, and virtual representations of various components of thesurgical system 100 (e.g., the virtual first patient tracker 136V, thevirtual second patient tracker 138V, the virtual digitization device106V, the virtual surgical tool 122V, one or more coordinate systems,and the like).

FIGS. 20A-20B also depict operation of one embodiment of the object/datamanager 180 of the GUI 172 of the CAD program 102. Here, the object/datamanager 180 may be used by the surgeon to, among other things, hide orun-hide various geometrical design objects GDO (compare FIG. 20A withFIG. 20B), view relationships associated with tracker assignmentadjustments 212 (e.g., which geometrical design objects GDO are fixed towhich patient trackers 136, 136), and view relationships associated withcompound type objects 198 (e.g., which primitive type objects 196 wereused for construction). In addition, in some embodiments, the contentsof the object/data manager 180 may be used to facilitate generatingcontextual options during a mixed approach 190, as is described ingreater detail below in connection with FIGS. 23B-23C.

FIGS. 21A-21B depict operation of one embodiment of the view manager 178of the GUI 172 of the CAD program 102. The view manager 178 allows thesurgeon to orientate the visualization VIZ depicted by the visualizationwindow 176 in different ways. In the illustrated embodiment, the viewmanager 178 comprises a use surgeon's view option 224, a use situs viewoption 226, a set surgeon's view option 228, a set situs view option230, a show all objects option 232, and an adjust view option 234.

The use surgeon's view option 224 has been selected in FIG. 21A, and theuse situs view option 226 has been selected in FIG. 21B. The adjust viewoption 234 can be selected to change the orientation of thevisualization VIZ depicted by the visualization window 176 in differentways, such as by rotating, panning, zooming, and the like (e.g., withgestures, with one or more control inputs 150, and the like). In someembodiments, the CAD program 102 is configured to enable the surgeon touse the adjust view option 234 to orientate the visualization VIZ basedon the position and/or orientation of the digitization device 106 (e.g.,by simulating a “camera” placed at the pointer tip 114).

If a desired orientation of the visualization VIZ has been establishedusing the adjust view option 234, the surgeon can subsequently use theset surgeon's view option 228 and/or the set situs view option 230 touse the desired orientation later during the procedure, and switchbetween the use surgeon's view option 224 and the use situs view option226 in a quick, efficient manner. In some embodiments, the CAD program102 may be configured to automatically determine one or moreorientations initially (e.g., based on tracked states SZ of one or moreof the trackers 120, 132, 136, 138, 142), and the surgeon cansubsequently modify the automatically determined orientation using theadjust view option 234. Other configurations are contemplated. The showall objects option 232 may be implemented in some embodiments of the CADprogram 102 to quickly and efficiently un-hide all of the geometricaldesign objects GDO, registered local virtual references LVR, calculatedvirtual references LVR, and/or virtual representations of the variouscomponents of the surgical system 100 rendered within the visualizationVIZ.

Referring now to FIGS. 22A-22B, additional aspects of the CAD program102 are shown in connection with using the new object arrangementsection 182 described above. As will be appreciated from the subsequentdescription below in connection with FIGS. 24A-24C, in some embodimentsthe CAD program 102 is configured to facilitate fixing (or“constraining”) one or more of the different types of geometrical designobjects GDO to the position and/or orientation of the virtualdigitization device 106V rendered in the visualization VIZ of thevirtual reference frame VRF. Put differently, the surgeon can constructnew geometrical design objects GDO (or move geometrical design objectsGDO arranged within the virtual reference frame VRF) by “snapping” themin predetermined orientations and/or positions relative to the virtualdigitizer reference point DRPV.

In some embodiments, this functionality can help the surgeon arrangeprimitive type objects 196 and/or compound type objects 198 using thetop-down approach 188, such as by fixing a selected geometrical designobject GDO in up to six degrees of freedom relative to the virtualdigitizer reference point DRPV before any local virtual references LVRfor the selected geometrical design object GDO have been established orregistered within the virtual reference frame VRF. The CAD program 102could then guide the surgeon through defining the selected geometricaldesign object GDO by sequentially establishing the requisite localvirtual references LVR which are needed to fully define the selectedgeometrical design object GDO. This concept is illustrated in anddescribed in greater detail below in connection with FIGS. 35-37.

In some embodiments, actuating the tool control input 128 could unfixthe selected geometrical design object GDO in one or more degrees offreedom based on subsequent (and sometimes successive) actuation of thetool control input 128. For example, actuating the tool control input128 a first time could establish a local virtual reference LVRassociated with the objet index OI of a plane PN, actuating the toolcontrol input 128 a second time could orientate the object index OIrelative to the CAD coordinate system CCS (e.g., by rotating the planePN about the object index OI), and a actuating the tool control input128 a third time could adjust the span length SL of the plane PN. Itwill be appreciated that the forgoing example is illustrative andnon-limiting, and other configurations are contemplated by the presentdisclosure.

Furthermore, in embodiments which utilize the HMD unit 140, a virtualrepresentation of the selected geometrical design object GDO may berendered overlaid onto the patient's anatomy relative to the target siteTS, which affords the surgeon with the ability to position the selectedgeometrical design object GDO with augmented reality (or mixed reality)by moving the digitization device 106 arbitrarily relative to the targetsite TS in ways which may or may not be associated with portions of thepatient's anatomy (e.g., with the pointer tip 114 positioned in air andnot contacting the anatomy). This functionality can also be utilizedwithout the HMD unit 140 while still affording the surgeon with theability to move the rendered geometrical design object GDO within thevisualization VIZ relative to existing geometrical design objects GDOarranged within the virtual reference frame VRF (e.g., relative to pointclouds PC; see FIGS. 24A-24C). Other configurations are contemplated.

With continued reference to FIGS. 22A-22B, the CAD program 102 may beconfigured to fix geometrical design objects GDO relative to the virtualdigitization device 106V (and, when used with the HMD unit 140, relativeto the digitization device 106) in different ways. More specifically,and as is described in greater detail below, the surgeon is able toselect between using an at pointer tip option 236, an offset frompointer tip option 238, and a with algorithm/from defined objects option240, each of which are described in greater detail below.

In FIG. 22A, the surgeon has selected the at pointer tip option 236 inorder to arrange a new geometrical design object GDO realized as a pointPT (landmark). Here, the visualization VIZ shows the point PT arrangedat the virtual digitizer reference point DRPV.

In FIG. 22B, the surgeon has selected the offset from pointer tip option238 to arrange a new geometrical design object GDO realized as a pointPT (landmark). Here, the visualization VIZ shows the point PT arrangedoffset from the virtual digitizer reference point DRPV at a distancethat can be adjusted by the surgeon. Thus, in addition to arranginggeometrical design objects GDO at locations of the anatomy relative tothe target site TS which correspond to physical contact of the pointertip 114, the CAD program may also be configured to allow geometricaldesign objects GDO to be arranged based on local virtual references LVRwhich are “projected” below the surface contacted by the pointer tip 114(e.g., projected into or through the bone). Other configurations arecontemplated.

Referring now to FIG. 23A, options for arranging a new geometricaldesign object GDO realized as a line segment LS are shown. Here, thesurgeon is able to select between a use pointer to digitize newlandmark(s) option 242 or a use defined object option 244. Here, the usepointer to digitize new landmark(s) option 242 may correspond to the atpointer tip option 238 and/or the offset from pointer tip option 238described above, and generally represents one embodiment of thebottom-up approach 186 described above. On the other hand, the usedefined objects option 244 may correspond to the with algorithm/formdefined objects option 240 described above, and generally represents oneembodiment of the top-down approach 188 described above.

In FIG. 23A, the use defined objects option 244 has been selected (e.g.,with the top-down approach 188), and a representation of the object/datamanager 180 is shown. Here, the surgeon is able to construct the newline segment LS using only some of the geometrical design objects GDO(or the local virtual references LVR and/or calculated virtualreferences CVR associated therewith). This is illustrated by some of theexisting geometrical design objects GDO being unavailable for thesurgeon to select (or “greyed out”), which may be determined by the CADprogram 102 contextually based on, among other things, the type ofsurgical step being planned, the type of surgical procedure to beexecuted, the types of different geometrical design objects GDO whichhave already been arranged within the virtual reference frame VRF, andthe like. Other configurations are contemplated.

Following from FIG. 23A, in FIG. 23B the surgeon has selected the usedefined objects option 244 to use an existing point PT (“Landmark 01”)to construct the line segment LS (e.g., by using the registered localvirtual reference LVR which defines “Landmark 01”). Here, the CADprogram switches from the top-down approach 188 to the mixed approach190, automatically, by rendering the line segment LS between thepreviously-selected point PT (“Landmark 01”) and the virtual digitizerreference point DRPV. While not shown in FIG. 23B, the surgeon couldcontinue with the mixed approach 190 by subsequently positioning thepointer tip 114 relative to the target site TS and establishing a newlocal virtual reference LVR which, when registered by the CAD program,would then fully define the line segment LS. Conversely, and as is shownin FIG. 23C, the surgeon has decided to continue with the top-downapproach 188 by selecting another existing point PT (“Landmark 02”) fromthe representation of the object/data manager 180 to construct and fullydefine the line segment LS (e.g., by using the registered local virtualreference LVR which defines “Landmark 02”).

Referring now to FIGS. 24A-24C, as noted above, different types ofgeometrical design objects GDO may be fixed relative to the virtual thevirtual digitizer reference point DRPV in different ways (e.g., with theat pointer tip option 236 or the offset from pointer tip option 238) to,among other things, help the surgeon initially orientate a newgeometrical design object (e.g., when following the top-down approach188).

In FIG. 24A the surgeon has selected the at pointer tip option 236 toarrange a new geometrical design object GDO realized as a plane PN usingthe top-down approach 188 and the surgeon is presented with examplecontextual options related to initially arranging and fixing a new planePN to the virtual digitization device 106V with the top-down approach188. Specifically, the surgeon is presented with an adjust sides option246, a perpendicular option 248, a parallel option 250, and aparallel/rotated option 252 to orientate and partially define anoctagonal plane PN. In some embodiments, additional options could bedisplayed (e.g., to initially set the span length SL). Otherconfigurations are contemplated.

In the embodiment illustrated in FIG. 24A, the surgeon has fixed anoctagonal plane PN perpendicular to the virtual digitization device 106Vby using the adjust sides 246 option to use eight sides, and by usingthe perpendicular option 248 to orientate the object index OI of theplane (e.g., the normal vector) perpendicular to the virtualdigitization device 106V.

In the embodiment illustrated in FIG. 24B, the surgeon has fixed anoctagonal plane PN parallel to the virtual digitization device 106V byusing the adjust sides 246 option to use eight sides, and by using theparallel option 250 to orientate the object index OI of the plane (e.g.,the normal vector) parallel to the virtual digitization device 106V(compare FIG. 24B to FIG. 24A).

In the embodiment illustrated in FIG. 24C, the surgeon has fixed anoctagonal plane PN parallel and rotated ninety-degrees (compare to FIG.24B) to the virtual digitization device 106V by using the adjust sides246 option to use eight sides, and by using the parallel/rotated option252 to orientate the object index OI of the plane (e.g., the normalvector) parallel to and rotated relative to the virtual digitizationdevice 106V (compare FIG. 24C to FIG. 24B).

While the illustrative examples described above in connection with FIGS.24A-24C involve fixing octagonal planes PN to the virtual digitizationdevice 106V, it will be appreciated that other types of differentgeometrical design objects GDO may be fixed to the virtual digitizationdevice 106V in various ways without departing from the scope of thepresent disclosure.

Referring now to FIG. 25A, continuing with the embodiment describedabove in connection with FIGS. 24A-24C, the surgeon has opted to arrangethe octagonal plane using the with algorithm/from defined objects option240, and has selected an existing point PT (“Landmark 01”) to positionthe object index OI of the octagonal plane PN (e.g., by using theregistered local virtual reference LVR which defines “Landmark 01”).Here, the CAD program switches to the mixed approach 190 such that theoctagonal plane PN rotates about the existing point PT (“Landmark 01”)to follow the orientation of the virtual digitization device 106V(compare FIGS. 25A-25B). While in the mixed approach 190, in addition torotating the octagonal plane PN about its object index OI relative tothe existing point PT to follow the virtual digitization device 106V,the CAD program 102 presents a prompt 254 to the surgeon to eitherselect additional objects from a representation of the object/datamanager 180 (e.g., local virtual references LVR and/or calculatedvirtual references CVR associated with existing geometrical designobjects GDO), or to digitize additional objects (e.g., establish andregister a new local virtual reference LVR).

In FIG. 25C, following with the mixed approach 190, the surgeon hasselected another existing point PT (“Landmark 02”) using therepresentation of the object/data manager 180 to continue defining theoctagonal plane PN (e.g., such as may be used to define the span lengthSL). Here in FIG. 25C, because the octagonal plane PN is not yet fullydefined, it still remains partially fixed to the virtual digitizationdevice 106V, and the prompt 254 continues to direct the surgeon toeither select additional objects or to digitize additional objects.

In FIG. 25D, the surgeon has opted to digitize a new local virtualreference LVR represented as a point PT (“Landmark 05”) arranged at oneof the corners of the octagonal plane PN, established such as bypositioning the pointer tip 114 at the target site TS and actuating thepointer control input 116 of the digitization device 106. Thenewly-created point PT (“Landmark 05”) is registered in the object/datamanager 180 and is shown as being associated with the first patienttracker 136 (“Tracker 01”).

Referring now to FIG. 26A, the surgeon has selected, using the existingobject arrangement section 184, a use to construct compound objectoption 254 selected using the octagonal plane PN (“Plane 1”) created asdescribed above in connection with FIGS. 20A-25D (specifically, see FIG.25D). In FIG. 26B, continuing with the mixed approach 190, the GUI 172of the CAD program 102 switches to the new object arrangement section182 in order to allow the surgeon to create a compound type object 198using the existing octagonal plane PN (“Plane 1”). Here, the surgeon hasopted to create an osteotomy plane OP by creating another octagonalplane PN which is parallel to and spaced from the existing octagonalplane PN (“Plane 01”) at a cut distance. Here too, it will beappreciated that the various arrangement, configuration, and navigationof the GUI 172 of the CAD program 102 described above in connection withFIGS. 20A-26B is non-limiting. Other configurations are contemplated.

As noted above, the surgical system 100, the CAD program 102, and thevarious methods and computer-implemented techniques of the presentdisclosure enable the surgeon to arrange different types of geometricaldesign objects GDO within the virtual reference frame VRF based on oneor more registered local virtual references LVR established using thedigitization device 106 in order to facilitate ad-hoc, intraoperativeplanning of surgical steps utilized during execution of surgicalprocedures. It will be appreciated that different surgical steps may beutilized for different surgical procedures, and that the same type ofsurgical procedure may be carried out using different types of surgicalsteps, arranged in different orders and/or sequences to accommodateparticular workflows, methodologies, and the like. In addition tofacilitating intraoperative planning of surgical steps, as notedpreviously, the surgical system 100, the CAD program 102, and thevarious methods and computer-implemented techniques of the presentdisclosure can also generate tool control data CZ which may be utilizedto control operation of the surgical tool 122 based on geometricaldesign objects GDO which were arranged intraoperatively within thevirtual reference frame VRF.

While the present disclosure is not limited to any one particularsurgical procedure, surgical step thereof, or workflow associatedtherewith, FIGS. 27-49 sequentially depict certain surgical steps whichmay be intraoperatively planned and executed, according to theembodiments of the surgical system 100 and the cad program 102 describedherein, in connection with performing femoral neck osteotomy. Here, thetarget site TS comprises the femoral neck FN of the femur F which, inthis example illustration, is to be resected such that the femoral headFH of the femur F can be removed and a prosthetic implant can besubsequently attached to the femur F (e.g., an artificial hip joint usedin connection with total hip arthroplasty; not shown herein).

Referring now to FIG. 27, the first patient tracker 136 is shownattached to the femur F at a location spaced from the target site TS andspaced from the digitization device 106 such that the pointer tip 114 isout of contact with the patient's anatomy (e.g., positioned in air).FIG. 27 also shows the visualization VIZ of the virtual reference frameVRF rendered by the CAD program 102, displayed such as by one of thedisplay units 148 of the navigation system 104. Also rendered in thevisualization VIZ is the virtual digitization device 106V, the virtualfirst patient tracker 136V (hereinafter “virtual patient tracker”), andthe CAD coordinate system CCS.

Continuing from FIG. 27 to FIG. 28, the pointer tip 114 of thedigitization device 106 is shown positioned about the femoral neck FN todemonstrate arranging a point cloud PC within the virtual referenceframe VRF. The resulting point cloud PC is shown rendered in thevisualization VIZ. In the illustrated embodiment, establishing andregistering the point cloud PC about the femoral neck FN is used to helpthe surgeon initially orientate the visualization VIZ, and is later usedto arrange an osteotomy plane OP.

Continuing from FIG. 28 to FIG. 29, the pointer tip 114 of thedigitization device 106 is shown positioned at the saddle point landmarkLM_SP of the femur F to demonstrate arranging a point (hereinafter,“saddle point PT_SP”) within the virtual reference frame VRF. Theresulting saddle point PT_SP is shown rendered in the visualization VIZ.

Continuing from FIG. 29 to FIG. 30, the pointer tip 114 of thedigitization device 106 is shown positioned at the trochanter minorlandmark LM_TM of the femur F to demonstrate arranging a point(hereinafter, “trochanter minor point PT_TM”) within the virtualreference frame VRF. The resulting trochanter minor point PT_TM is shownrendered in the visualization VIZ.

Continuing from FIG. 30 to FIG. 31, the pointer tip 114 of thedigitization device 106 is shown positioned at the medial epicondylelandmark LM_ME of the femur F to demonstrate arranging a point(hereinafter, “medial epicondyle point PT_ME”) within the virtualreference frame VRF. The resulting medial epicondyle point PT_LE” isshown rendered in the visualization VIZ. Further, in FIG. 31, thevisualization VIZ depicts a line segment (hereinafter, “first linesegment LS_1”) arranged extending between the saddle point PT_SP and thetrochanter minor point PT_TM.

Continuing from FIG. 31 to FIG. 32, the pointer tip 114 of thedigitization device 106 is shown positioned at the lateral epicondylelandmark LM_LE of the femur F to demonstrate arranging a point(hereinafter, “lateral epicondyle point PT_LE”) within the virtualreference frame VRF. The resulting lateral epicondyle point PT_LE isshown rendered in the visualization VIZ. Further, in FIG. 32, thevisualization VIZ depicts a line segment (hereinafter, “second linesegment LS_2”) arranged extending between the medial epicondyle pointPT_ME and the lateral epicondyle point PT_LE. Contextually, the secondline segment LS_2 is the transepicondylar axis of the femur F.

Continuing from FIG. 32 to FIG. 33, the visualization VIZ depicts apoint PT arranged along the second line segment LS_2 midway between themedial epicondyle point PT_ME and the lateral epicondyle point PT_LE.

Continuing from FIG. 33 to FIG. 34, the visualization VIZ depicts a linesegment (hereinafter, “third line segment LS_3”) arranged extendingbetween the point PT and the saddle point PT_SP. Contextually, the thirdline segment LS_3 is the anatomical axis of the femur F.

Continuing from FIG. 34 to FIG. 35, the visualization VIZ depicts anoctagonal plane (hereinafter, “first plane PN_1”) fixed to the virtualdigitization device 106V in the parallel option 250 (as described abovein connection with FIG. 24C) so as to enable the surgeon to arrange thefirst plane PN_1 within the virtual reference frame VRF by moving thedigitization device 106 relative to the target site TS. In FIG. 34 (andalso in FIGS. 35-40A and 41-42), the object index OI of the first planePN_1 is not illustrated as a coordinate system with a normal vector aspreviously described and illustrated; rather, the object index OI isrepresented by the intersection of two dash-dot-dash lines arrangedperpendicular to each other and extending between corners of theoctagonal first plane PN_1 for clarity and to help illustrate movementand adjustment of the first plane PN_1 (e.g., as may be demonstrated bysuccessively viewing FIGS. 35-40A).

Continuing from FIG. 35 to FIG. 36, the visualization VIZ has beenenlarged (e.g., by using the adjust view option 234 described above tozoom and pan), and depicts the first plane PN_1 still fixed to thevirtual digitization device 106V but moved to a different pose based oncorresponding movement of the digitization device 106 relative to thetarget site TS (compare FIG. 36 with FIG. 35).

Continuing from FIG. 36 to FIG. 37, the visualization VIZ depicts thefirst plane PN_1 still (partially) fixed to the virtual digitizationdevice 106V but is moved to yet another different pose based oncorresponding movement of the digitization device 106 relative to thetarget site TS (compare FIG. 37 with FIG. 36). Here in FIG. 37, theobject index OI of the first plane PN_1 is also fixed to and can rotateabout the saddle point PT_SP based on movement of the digitizationdevice 106 (as described above in connection with FIGS. 25A-25B). Thisallows the surgeon to visualize and thereby orientate the first planePN_1 relative to the target site TS using the digitization device 106,and the surgeon can unfix the first plane PN_1 from the virtualdigitization device 106V (e.g., by actuating the pointer control input116) and subsequently the GUI 172 of the CAD program 102 to fully-definethe first plane PN_1. As shown in FIG. 37, the first plane PN1 is shownorientated at a first angle A1 with respect to the third line segmentLS_3 (the anatomical axis of the femur F), such as may be determinedwith the angle measurements 200 functionality of the CAD program 102described previously.

Continuing from FIG. 37 to FIG. 38, the visualization VIZ depicts thefirst plane PN_1 with its object index OI still fixed to the saddlepoint PT_SP, but having been subsequently orientated to a second angleA2 (e.g., a larger angle; compare FIG. 38 with FIG. 37) with respect tothe third line segment LS_3 (the anatomical axis of the femur F). Thischange in arrangement may likewise have been carried out using the anglemeasurements 200 functionality of the CAD program 102 describedpreviously.

Continuing from FIG. 38 to FIG. 39, the visualization VIZ depicts thefirst plane PN_1 in the same pose, but is shown at at a first distanceD1 measured with respect to the trochanter minor point PT_TM. Here, thefirst distance D1 may have been determined with the distancemeasurements 202 functionality of the CAD program 102 describedpreviously.

Continuing from FIG. 39 to FIG. 40A, the visualization depicts the firstplane PN_1 in a different pose after having been moved to a seconddistance D2 (e.g., a smaller distance; compare FIG. 40A with FIG. 39)with respect to the trochanter minor point PT_TM. This change inarrangement may likewise have been carried out using the distancemeasurements 202 functionality of the CAD program 102 describedpreviously. Furthermore, as is demonstrated in FIG. 40A, the secondangle A2 is maintained with respect to the third line segment LS_3 (theanatomical axis of the femur F). Thus, FIG. 40A depicts the first planePN_1 as fully-defined and arranged within the virtual reference frameVRF.

Continuing from FIG. 40A to FIG. 40B, a portion of the femur F is shownwith the first patient tracker 136 attached thereto as viewed by thesurgeon through the HMD unit 140 (as described above in connection withFIG. 19F). Here in FIG. 40B, because the HMD unit 140 is able to renderthe visualization VIZ based on the visualization data VZ generated usingthe tracked states SZ monitored with the navigation system 104, thesurgeon is able to view the fully-defined first plane PN_1 renderedoverlaid onto the femur F with augmented reality (or “mixed reality”).Further, the saddle point PT_SP of the visualization VIZ is alsorendered overlaid onto the saddle point landmark LM_SP of the femur F.

Continuing now from FIG. 40A to FIG. 41, the visualization VIZ depictsthe first plane PN_1 in the same pose, and also depicts a secondoctagonal plane (hereinafter, “second plane PN_2”) arranged parallel toand spaced from the first plane PN_1 at a first cut distance C1. Thefirst plane PN_1 and the second plane PN_2 here are configured as acompound type object 198 realized as an osteotomy plane OP (as describedabove in connection with FIG. 14). Here, the first cut distance C1 mayhave been determined with the distance measurements 202 functionality ofthe CAD program 102 described previously.

Continuing from FIG. 41 to FIG. 42, the visualization VIZ depicts thefirst plane PN_1 in the same pose, but depicts the second plane PN_2 ina different pose. Here, while still parallel to and generally alignedwith the first plane PN_1, the second plane PN_2 has been moved relativeto the first plane PN_1 to a second cut distance C2 (e.g. a smaller cutdistance; compare FIG. 42 with FIG. 41). Here too, the second cutdistance C2 may have been carried out using the distance measurements202 functionality of the CAD program 102 described previously.

Continuing from FIG. 42 to FIG. 43, the visualization VIZ depicts abounding volume BV generated, such as via the algorithms module 174described above, between the first and second planes PN_1, PN_2. Here,this bounding volume BV may be used to help clip a surface generatedfrom the point cloud PC (e.g., a triangle mesh TM), and/or may be usedto calculate a milling volume MV, define one or more virtual boundaries216, and/or generate tool control data CZ in some embodiments.

Continuing from FIG. 43 to FIG. 44, the pointer tip 114 of thedigitization device 106 is shown positioned differently about thefemoral neck FN (compare FIG. 44 with FIG. 28) to demonstrate arrangingadditional points PT which are to be merged with the point cloud PCwithin the virtual reference frame VRF. The resulting merged point cloudPC is shown rendered in the visualization VIZ. Here, arrangingadditional points PT may be bounded by the bounding volume BV defined bythe first and second planes PN_1, PN_2 in order to form a more accuratetriangle mesh TM used to generate a milling volume MV. Put differently,points PT established where the pointer tip 114 is positioned outside ofthe bounding volume BV may be ignored or otherwise not registered by theCAD program 102.

Continuing from FIG. 44 to FIG. 45, the visualization VIZ shows arendered milling volume MV generated using the point cloud PC and thebounding volume BV illustrated in FIG. 44. Here, the milling volume MVmay be generated by the CAD program 102 using various aspects of thealgorithm module 174 described above (e.g., calculating a triangle meshTM with the point cloud PC and clipping the triangle mesh TM with thefirst and second planes PN_1, PN_2).

Continuing from FIG. 45 to FIG. 46A, the surgical tool 122 is shownspaced from the target site TS such that the energy applicator 124 isout of contact with the patient's anatomy (e.g., positioned in air). Thevirtual surgical tool 122V is rendered in the visualization VIZ adjacentto the milling volume MV. FIG. 46A generally depicts the target site TSjust before the femoral neck FN is to be resected during execution ofthe femoral neck osteotomy.

Continuing from FIG. 46A to FIG. 46B, similar to FIG. 40B describedabove, a portion of the femur F is shown with the first patient tracker136 attached thereto as viewed by the surgeon through the HMD unit 140.Here in FIG. 46B, because the HMD unit 140 is able to render thevisualization VIZ based on the visualization data VZ generated using thetracked states SZ monitored with the navigation system 104, the surgeonis able to view the milling volume MV rendered overlaid onto the femur Fwith augmented reality (or “mixed reality”). Further, the saddle pointPT_SP of the visualization VIZ is also rendered overlaid onto the saddlepoint landmark LM_SP of the femur F.

Continuing now from FIG. 46A to FIG. 47, the surgical tool 122 is shownwith the energy applicator 124 beginning resection of the femoral headFH along the femoral neck FN at the target site TS, and thevisualization VIZ depicts the relative pose of the virtual surgical tool122V which is shown positioned within the milling volume MV.

Continuing from FIG. 47 to FIG. 48, the surgical tool 122 is shown withthe energy applicator 124 continuing resection of the femoral head FHalong the femoral neck FN at the target site TS, and the visualizationVIZ depicts the relative pose of the virtual surgical tool 122V which isshown positioned further within the milling volume MV (compare FIG. 48with FIG. 47).

Continuing from FIG. 48 to FIG. 49, the surgical tool 122 is shownspaced from the target site TS after having completed resection of thefemoral head FH, which is shown spaced from the femur F.

In this way, the surgical system 100, the CAD program 102, and thevarious methods and computer-implemented techniques of the presentdisclosure enable the surgeon to arrange different types of geometricaldesign objects GDO within the virtual reference frame VRF based on oneor more registered local virtual references LVR established using thedigitization device 106 in order to facilitate ad-hoc, intraoperativeplanning of a variety of different surgical steps which may be utilizedduring the execution of a broad array of different surgical procedures.Specifically, it will be appreciated that various different geometricaldesign objects GDO can be arranged within the virtual reference frameVRF based on registered local virtual references LVR and/or calculatedvirtual references CVR to intraoperatively plan surgical steps in anumber of different ways, and/or for a number of different purposes.

Thus, the surgeon can intraoperatively identify anatomical features withor without reliance on preoperatively-acquired imaging of the targetsite TS. In addition, the surgeon can intraoperatively arrangegeometrical design objects GDO relative to those identified anatomicalfeatures and, based on the arrangement, intraoperatively visualizeand/or orientate geometrical design objects GDO in ways which representcuts to be made during execution, holes to be drilled during execution,and/or milling volumes to be removed during execution.

Furthermore, the surgeon can use the intraoperatively-arrangedgeometrical design objects GDO to generate cool control data CZ usedduring execution, such as may be used to control the position and/oroperation of the surgical tool 122 relative to the target site TS usingintraoperatively-defined milling volumes MV, virtual boundaries 216, andthe like. Further still, the surgeon can manipulate, position, and/orvisualize geometrical design objects GDO, the virtual digitizationdevice 106V, the virtual surgical tool 122V, the virtual patienttrackers 136V, 138V, and the like within the visualization VIZ of thevirtual reference frame VRF in near-real time using the display unit148. Moreover, using the HMD unit 140, the surgeon can visualizegeometrical design objects GDO rendered overlaid onto the patient'sanatomy relative to the target site TS in near-real time using augmentedor mixed reality.

It will be further appreciated that the terms “include,” “includes,” and“including” have the same meaning as the terms “comprise,” “comprises,”and “comprising.” Moreover, it will be appreciated that terms such as“first,” “second,” “third,” and the like are used herein todifferentiate certain structural features and components for thenon-limiting, illustrative purposes of clarity and consistency.

Several configurations have been discussed in the foregoing description.However, the configurations discussed herein are not intended to beexhaustive or limit the invention to any particular form. Theterminology which has been used is intended to be in the nature of wordsof description rather than of limitation. Many modifications andvariations are possible in light of the above teachings and theinvention may be practiced otherwise than as specifically described.

The invention is intended to be defined in the independent claims, withspecific features laid out in the dependent claims, wherein thesubject-matter of a claim dependent from one independent claim can alsobe implemented in connection with another independent claim.

The invention claimed is:
 1. A surgical system for facilitating ad-hoc intraoperative planning of a surgical step to be performed at a target site, the surgical system comprising: a digitization device comprising a pointer tip and one or more control inputs and the digitization device being configured to intraoperatively facilitate establishment of one or more local virtual references relative to the target site, wherein the one or more local virtual references are established relative to the pointer tip in response to actuation of the one or more control inputs; a navigation system configured to track states of the digitization device; and a computing device coupled to the navigation system and comprising one or more processors and a non-transitory storage medium having stored thereon a computer-aided design (CAD) program that when executed by the one or more processors is configured to: generate a virtual reference frame; receive the tracked states of the digitization device from the navigation system; register the one or more local virtual references within the virtual reference frame; render, within the virtual reference frame, a virtual representation of the pointer tip of the digitization device having a position and orientation of the pointer tip derived from the tracked states of the digitization device; and enable arrangement of different types of geometrical design objects within the virtual reference frame relative to one or more registered local virtual references to intraoperatively plan the surgical step, and wherein to enable arrangement of different types of geometrical design objects, the CAD program is further configured to: enable one or more of the geometrical design objects to be selected and temporarily fixed to the virtual representation of the pointer tip within the virtual reference frame such that the one or more geometrical design objects that are temporarily fixed correspondingly follow changes in position and orientation of the virtual representation of the pointer tip within the virtual reference frame; and in response to actuation of the one or more control inputs of the digitization device, enable the one or more geometrical design objects to become unfixed from the virtual representation of the pointer tip and placed in the virtual reference frame relative to the one or more local virtual references.
 2. The surgical system as set forth in claim 1, wherein the CAD program is further configured to enable arrangement of the different types of geometrical design objects within the virtual reference frame by: constructing a new geometrical design object from registered local virtual references sequentially-established with the digitization device; constructing a new geometrical design object from a previously-constructed geometrical design object arranged within the virtual reference frame; and/or adjusting a previously-constructed geometrical design object arranged within the virtual reference frame.
 3. The surgical system as set forth in claim 1, wherein the different types of geometrical design objects comprise: a first type of the geometrical design object comprising one of: a point, a line, a plane, or a volume; and a second type of geometrical design object comprising a different one of: a point, a line, a plane, or a volume.
 4. The surgical system as set forth in claim 1, wherein the CAD program is further configured to enable arrangement of the different types of geometrical design objects within the virtual reference frame based on geometric relationships with respect to one or more of: a registered local virtual reference, and/or a calculated virtual reference derived from one or more registered local virtual references.
 5. The surgical system as set forth in claim 1, wherein the CAD program is further configured to enable construction of a compound object from one or more geometrical design objects arranged within the virtual reference frame.
 6. The surgical system as set forth in claim 1, further comprising a patient tracker adapted for attachment relative to the target site; wherein the navigation system is configured to track states of the patient tracker; wherein the CAD program is further configured to render, within the virtual reference frame, a virtual representation of the patient tracker having a position and/or orientation derived from the tracked states of the patient tracker; and fix one or more registered local virtual references to the virtual representation of the patient tracker within the virtual reference frame.
 7. The surgical system as set forth in claim 1, further comprising a surgical tool; wherein the navigation system is configured to track states of the surgical tool; and wherein the CAD program is further configured to render, within the virtual reference frame, a virtual representation of the surgical tool having a position and/or orientation derived from the tracked states of the surgical tool.
 8. The surgical system as set forth in claim 7, wherein the CAD program is further configured to facilitate establishing a virtual boundary with one or more geometrical design objects arranged within the virtual reference frame to control movement and/or operation of the surgical tool relative to the target site.
 9. The surgical system as set forth in claim 1, wherein the CAD program is further configured to enable construction of a virtual implant model from one or more geometrical design objects arranged within the virtual reference frame.
 10. The surgical system as set forth in claim 9, further comprising an implant manufacturing device coupled to the computing device and configured to intraoperatively generate an implant based on the virtual implant model using an additive manufacturing technique.
 11. The surgical system as set forth in claim 1, further comprising a head-mountable display unit in communication with the computing device and being configured to: receive data generated with the CAD program; and based on the received data, render a visualization of the virtual reference frame, the one or more registered local virtual references, and/or the different types of geometrical design objects arranged within the virtual reference frame.
 12. The surgical system as set forth in claim 11, further comprising: a patient tracker adapted for attachment relative to the target site; and a display unit tracker coupled to the head-mountable display unit; wherein the navigation system is configured to: track states of the display unit tracker; track states of the patient tracker; and register the visualization to the target site; and wherein the head-mountable display unit is configured to render the registered visualization in augmented reality and/or mixed reality such that the virtual reference frame, the one or more registered local virtual references, and the different types of geometrical design objects are visible relative to the target site for any given state of the display unit tracker.
 13. A computer-assisted method for facilitating ad-hoc intraoperative planning of a surgical step to be performed at a target site using a surgical system comprising a digitization device comprising a pointer tip and one or more control inputs, a navigation system, and a computing device coupled to the navigation system and comprising one or more processors and a non-transitory storage medium having stored thereon a computer-aided design (CAD) program being executable by the one or more processors, the method comprising: tracking states of the digitization device with the navigation system; intraoperatively establishing one or more local virtual references relative to the target site with the digitization device, wherein the one or more local virtual references are established relative to the pointer tip in response to actuation of the one or more control inputs; generating a virtual reference frame with the CAD program; registering the one or more local virtual references within the virtual reference frame with the CAD program; rendering, within the virtual reference frame, a virtual representation of the pointer tip of the digitization device having a position and orientation of the pointer tip derived from the tracked states of the digitization device; and arranging different types of geometrical design objects within the virtual reference frame relative to one or more registered local virtual references with the CAD program to intraoperatively plan the surgical step by: selecting and temporarily fixing one or more of the geometrical design objects to the virtual representation of the pointer tip within the virtual reference frame such that the one or more geometrical design objects that are temporarily fixed correspondingly follow changes in position and orientation of the virtual representation of the pointer tip within the virtual reference frame; and in response to actuation of the one or more control inputs of the digitization device, unfixing the one or more geometrical design objects from the virtual representation of the pointer tip and placing the one or more geometrical design objects in the virtual reference frame relative to the one or more local virtual references.
 14. The method as set forth in claim 13, wherein arranging different types of geometrical design objects within the virtual reference frame with the CAD program further comprises: constructing a new geometrical design object from registered local virtual references sequentially-established with the digitization device; constructing a new geometrical design object from a previously-constructed geometrical design object arranged within the virtual reference frame; and/or adjusting a previously-constructed geometrical design object arranged within the virtual reference frame.
 15. The method as set forth in claim 13, further comprising: deriving, with the CAD program, one or more calculated virtual references from one or more registered local virtual references; and arranging, with the CAD program, the different types of geometrical design objects within the virtual reference frame based on geometric relationships with respect to one or more registered local virtual references and/or calculated virtual references.
 16. The method as set forth in claim 13, further comprising constructing, with the CAD program, a compound object from one or more geometrical design objects arranged within the virtual reference frame.
 17. The method as set forth in claim 13, wherein the surgical system further comprises a patient tracker adapted for attachment relative to the target site, wherein the navigation system is configured to track states of the patient tracker, and wherein the method further comprises: rendering, with the CAD program, a virtual representation of the patient tracker within the virtual reference frame having a position and/or orientation derived from the tracked states of the patient tracker; and fixing, with the CAD program, one or more registered local virtual references to the virtual representation of the patient tracker within the virtual reference frame.
 18. The method as set forth in claim 13, wherein the surgical system further comprises a surgical tool, wherein the navigation system is configured to track states of the surgical tool, and wherein the method further comprises rendering, with the CAD program, a virtual representation of the surgical tool within the virtual reference frame having a position and/or orientation derived from the tracked states of the surgical tool.
 19. The method as set forth in claim 18, further comprising establishing, with the CAD program, a virtual boundary with one or more geometrical design objects arranged within the virtual reference frame to control movement and/or operation of the surgical tool relative to the target site.
 20. The method as set forth in claim 13, further comprising constructing, with the CAD program, a virtual implant model from one or more geometrical design objects arranged within the virtual reference frame.
 21. The method as set forth in claim 13, wherein the surgical system further comprises a head-mountable display unit in communication with the computing device and wherein the method further comprises: receiving, with the head-mountable display unit, data generated by the CAD program; and rendering, with the head-mountable display unit and based on the received data, a visualization of the virtual reference frame, the one or more registered local virtual references, and/or the different types of geometrical design objects arranged within the virtual reference frame.
 22. The method as set forth in claim 21, wherein the surgical system further comprises a patient tracker adapted for attachment relative to the target site and a display unit tracker coupled to the head-mountable display unit, wherein the navigation system is configured to track states of the display unit tracker and to track states of the patient tracker and to register the visualization to the target site, and wherein the method further comprises rendering, with the head-mountable display unit, the registered visualization in augmented reality and/or mixed reality such that the virtual reference frame, the one or more registered local virtual references, and the different types of geometrical design objects are visible relative to the target site for any given state of the display unit tracker.
 23. A non-transitory storage medium having stored thereon a computer-aided design (CAD) program being configured to facilitate ad-hoc intraoperative planning of a surgical step to be performed at a target site, the CAD program being utilized with a surgical system comprising a digitization device including a pointer tip and one or more control inputs and the digitization device being configured to intraoperatively facilitate establishment of one or more local virtual references relative to the target site, wherein the one or more local virtual references are established relative to the pointer tip in response to actuation of the one or more control inputs, and a navigation system configured to track states of the digitization device, the CAD program when executed by one or more processors is configured to: generate a virtual reference frame; receive tracked states of the digitization device from a navigation system; register the one or more local virtual references established relative to the target site within the virtual reference frame; render, within the virtual reference frame, a virtual representation of the pointer tip of the digitization device having a position and orientation of the pointer tip derived from the tracked states of the digitization device; enable arrangement of different types of geometrical design objects within the virtual reference frame relative to one or more registered local virtual references to intraoperatively plan the surgical step, and wherein to enable arrangement of different types of geometrical design objects, the CAD program is further configured to; enable one or more of the geometrical design objects to be selected and temporarily fixed to the virtual representation of the pointer tip within the virtual reference frame such that the one or more geometrical design objects that are temporarily fixed correspondingly follow changes in position and orientation of the virtual representation of the pointer tip within the virtual reference frame; and in response to actuation of the one or more control inputs of the digitization device, enable the one or more geometrical design objects to become unfixed from the virtual representation of the pointer tip and placed in the virtual reference frame relative to the one or more local virtual references.
 24. The non-transitory storage medium as set forth in claim 23, wherein the CAD program is further configured to enable arrangement of the different types of geometrical design objects within the virtual reference frame by: constructing a new geometrical design object from sequentially-established registered local virtual references; constructing a new geometrical design object from a previously-constructed geometrical design object arranged within the virtual reference frame; and/or adjusting a previously-constructed geometrical design object arranged within the virtual reference frame.
 25. The non-transitory storage medium as set forth in claim 23, wherein the different types of geometrical design objects comprise: a first type of the geometrical design object comprising one of: a point, a line, a plane, or a volume; and a second type of geometrical design object comprising a different one of: a point, a line, a plane, or a volume.
 26. The non-transitory storage medium as set forth in claim 23, wherein the CAD program is further configured to enable arrangement of the different types of geometrical design objects within the reference frame based on geometric relationships with respect to one or more of: a registered local virtual reference, and/or a calculated virtual reference derived from one or more registered local virtual references.
 27. The non-transitory storage medium as set forth in claim 23, wherein the CAD program is further configured to enable construction of a compound object from one or more geometrical design objects arranged within the virtual reference frame.
 28. The non-transitory storage medium as set forth in claim 23, wherein the CAD program is further configured to enable construction of a virtual implant model from one or more geometrical design objects arranged within the virtual reference frame. 